[llvm] [CodeGen][AMDGPU] Replace isBasicBlockPrologue with a MachineInstr::BBProlog flag (PR #225433)
Gabriel Baraldi via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 25 05:46:17 PDT 2026
https://github.com/gbaraldi updated https://github.com/llvm/llvm-project/pull/225433
>From 3105860c7556d223172f8cd6e882ea1683e22f13 Mon Sep 17 00:00:00 2001
From: Gabriel Baraldi <baraldigabriel at gmail.com>
Date: Tue, 22 Sep 2026 15:24:16 +0000
Subject: [PATCH 1/3] [CodeGen][AMDGPU] Replace isBasicBlockPrologue with a
MachineInstr::BBProlog flag
TargetInstrInfo::isBasicBlockPrologue asked the target to infer, per
instruction and per register being allocated, whether an instruction at
the top of a block belongs to the block prolog. AMDGPU implemented this
by pattern matching everything the register allocator might have
inserted ahead of the exec restore (SGPR spills, WWM spills, implicit
defs, LRSplit copies) plus any EXEC write, and by sending all SGPR
insertions to MBB.begin(). Every new kind of RA-inserted instruction had
to be taught to the check; rematerialized instructions were not, which
let the vgpr-regalloc pipeline place copies and spills above the exec
restore of an if/else Flow block (#222368).
Represent the prolog explicitly instead:
* Add MachineInstr::BBProlog (MIR: bb-prolog). An instruction carrying it
must stay ahead of everything later inserted at the block entry.
* MachineBasicBlock::SkipPHIsAndLabels / SkipPHIsLabelsAndDebug skip
flagged instructions. The Register parameter is gone, as are the
TargetInstrInfo hook and SIInstrInfo::canAddToBBProlog. MachineSink,
RegAllocFast and SIFixSGPRCopies test the flag directly.
* Code that inserts in front of an existing instruction calls
MachineBasicBlock::inheritBBProlog, which marks the new instructions
when the next non-debug instruction is a prolog instruction: spills,
reloads, split copies, rematerialization and memory operand folding in
the register allocators, PHI elimination, and AMDGPU's SGPR spill
lowering.
* The MachineVerifier checks that, until register allocation is done,
prolog instructions form a block prefix, only preceded by PHIs, labels
and debug instructions. A missed insertion site fails verification.
* AMDGPU sets the flag where it creates prolog instructions at the block
start: SI_END_CF at instruction selection, the exec restores emitted
by SILowerControlFlow, and the exec writes SIWholeQuadMode emits at
block entry.
The sgpr-regalloc pipeline no longer inserts at MBB.begin(); its copies,
remats and reloads now follow the exec restore like everything else.
Scalar instructions do not depend on EXEC, so this only reorders code.
Test updates follow in separate commits.
Fixes #222368
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply at anthropic.com>
---
llvm/include/llvm/CodeGen/MachineBasicBlock.h | 21 ++--
llvm/include/llvm/CodeGen/MachineInstr.h | 4 +-
llvm/include/llvm/CodeGen/TargetInstrInfo.h | 9 --
.../CodeGen/FixupStatepointCallerSaved.cpp | 3 +-
llvm/lib/CodeGen/InlineSpiller.cpp | 4 +-
llvm/lib/CodeGen/LiveRangeEdit.cpp | 2 +
llvm/lib/CodeGen/MIRParser/MILexer.cpp | 1 +
llvm/lib/CodeGen/MIRParser/MILexer.h | 1 +
llvm/lib/CodeGen/MIRParser/MIParser.cpp | 5 +-
llvm/lib/CodeGen/MIRPrinter.cpp | 2 +
llvm/lib/CodeGen/MachineBasicBlock.cpp | 23 ++--
llvm/lib/CodeGen/MachineInstr.cpp | 2 +
llvm/lib/CodeGen/MachineSink.cpp | 4 +-
llvm/lib/CodeGen/MachineVerifier.cpp | 16 +++
llvm/lib/CodeGen/PHIElimination.cpp | 2 +
llvm/lib/CodeGen/RegAllocFast.cpp | 6 +-
llvm/lib/CodeGen/RegAllocGreedy.cpp | 3 +-
llvm/lib/CodeGen/SplitKit.cpp | 5 +-
llvm/lib/CodeGen/TargetInstrInfo.cpp | 6 ++
.../AMDGPU/AMDGPUInstructionSelector.cpp | 7 +-
llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp | 11 +-
llvm/lib/Target/AMDGPU/SIISelLowering.cpp | 6 ++
llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 45 --------
llvm/lib/Target/AMDGPU/SIInstrInfo.h | 13 ++-
llvm/lib/Target/AMDGPU/SIInstructions.td | 1 +
llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp | 5 +
llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp | 2 +
llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 6 ++
llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp | 16 ++-
.../AMDGPU/sgpr-remat-in-bb-prolog.mir | 82 ++++++++++++++
.../CodeGen/MIR/AMDGPU/bb-prolog-flag.mir | 17 +++
.../test/CodeGen/MIR/AMDGPU/lr-split-flag.mir | 3 +-
.../test/MachineVerifier/AMDGPU/bb-prolog.mir | 43 ++++++++
.../CodeGen/MachineBasicBlockTest.cpp | 101 ++++++++++++++++++
llvm/utils/UpdateTestChecks/mir.py | 2 +-
35 files changed, 379 insertions(+), 100 deletions(-)
create mode 100644 llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir
create mode 100644 llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir
create mode 100644 llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir
diff --git a/llvm/include/llvm/CodeGen/MachineBasicBlock.h b/llvm/include/llvm/CodeGen/MachineBasicBlock.h
index 215562968158f9..0247cca6472f88 100644
--- a/llvm/include/llvm/CodeGen/MachineBasicBlock.h
+++ b/llvm/include/llvm/CodeGen/MachineBasicBlock.h
@@ -903,19 +903,24 @@ class MachineBasicBlock
return const_cast<MachineBasicBlock *>(this)->getFirstNonPHI();
}
- /// Return the first instruction in MBB after I that is not a PHI or a label.
- /// This is the correct point to insert lowered copies at the beginning of a
- /// basic block that must be before any debugging information.
+ /// Return the first instruction in MBB after I that is not a PHI, a label or
+ /// a block prolog instruction. This is the correct point to insert lowered
+ /// copies at the beginning of a basic block that must be before any debugging
+ /// information.
LLVM_ABI iterator SkipPHIsAndLabels(iterator I);
- /// Return the first instruction in MBB after I that is not a PHI, label or
- /// debug. This is the correct point to insert copies at the beginning of a
- /// basic block. \p Reg is the register being used by a spill or defined for a
- /// restore/split during register allocation.
+ /// Return the first instruction in MBB after I that is not a PHI, label,
+ /// debug or block prolog instruction. This is the correct point to insert
+ /// copies at the beginning of a basic block.
LLVM_ABI iterator SkipPHIsLabelsAndDebug(iterator I,
- Register Reg = Register(),
bool SkipPseudoOp = true);
+ /// Instructions [\p Begin, \p End) were inserted in front of \p End. If that
+ /// is inside the block prolog (the next instruction that is not a label,
+ /// debug instruction or pseudo probe is a prolog instruction), mark them as
+ /// prolog instructions too.
+ LLVM_ABI void inheritBBProlog(iterator Begin, iterator End);
+
/// Returns an iterator to the first terminator instruction of this basic
/// block. If a terminator does not exist, it returns end().
LLVM_ABI iterator getFirstTerminator();
diff --git a/llvm/include/llvm/CodeGen/MachineInstr.h b/llvm/include/llvm/CodeGen/MachineInstr.h
index a18ee9bde66b78..561db4c652d101 100644
--- a/llvm/include/llvm/CodeGen/MachineInstr.h
+++ b/llvm/include/llvm/CodeGen/MachineInstr.h
@@ -129,8 +129,10 @@ class MachineInstr
InBounds = 1 << 22, // Pointer arithmetic remains inbounds.
// Implies NoUSWrap.
LRSplit = 1 << 23, // Instruction for live range split.
- NonNull = 1 << 24 // Address space cast source is not the null
+ NonNull = 1 << 24, // Address space cast source is not the null
// value of the source address space.
+ BBProlog = 1 << 25 // Part of the basic block prolog: must stay
+ // ahead of everything inserted at block entry.
};
static constexpr uint32_t getPoisonGeneratingFlags() {
diff --git a/llvm/include/llvm/CodeGen/TargetInstrInfo.h b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
index b013511d333135..94db7f162967f5 100644
--- a/llvm/include/llvm/CodeGen/TargetInstrInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
@@ -2190,15 +2190,6 @@ class LLVM_ABI TargetInstrInfo : public MCInstrInfo {
return Inst.isReturn() && Inst.isCall();
}
- /// True if the instruction is bound to the top of its basic block and no
- /// other instructions shall be inserted before it. This can be implemented
- /// to prevent register allocator to insert spills for \p Reg before such
- /// instructions.
- virtual bool isBasicBlockPrologue(const MachineInstr &MI,
- Register Reg = Register()) const {
- return false;
- }
-
/// Allows targets to use appropriate copy instruction while spilitting live
/// range of a register in register allocation.
virtual unsigned getLiveRangeSplitOpcode(Register Reg,
diff --git a/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp b/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp
index 8d2f9bfb6c5cbb..2be3def1b09d04 100644
--- a/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp
+++ b/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp
@@ -455,8 +455,7 @@ class StatepointState {
<< RegToSlotIdx[Reg] << " after statepoint\n");
if (EHPad && RC.tryRecordReload(Reg, RegToSlotIdx[Reg], EHPad)) {
- auto EHPadInsertPoint =
- EHPad->SkipPHIsLabelsAndDebug(EHPad->begin(), Reg);
+ auto EHPadInsertPoint = EHPad->SkipPHIsLabelsAndDebug(EHPad->begin());
insertReloadBefore(Reg, EHPadInsertPoint, EHPad);
LLVM_DEBUG(dbgs() << "...also reload at EHPad "
<< printMBBReference(*EHPad) << "\n");
diff --git a/llvm/lib/CodeGen/InlineSpiller.cpp b/llvm/lib/CodeGen/InlineSpiller.cpp
index f3682a2e24808a..01f6bd0f35dcf2 100644
--- a/llvm/lib/CodeGen/InlineSpiller.cpp
+++ b/llvm/lib/CodeGen/InlineSpiller.cpp
@@ -457,7 +457,7 @@ bool InlineSpiller::hoistSpillInsideBB(LiveInterval &SpillLI,
MachineBasicBlock *MBB = DefMBB;
MachineBasicBlock::iterator MII;
if (SrcVNI->isPHIDef())
- MII = MBB->SkipPHIsLabelsAndDebug(MBB->begin(), SrcReg);
+ MII = MBB->SkipPHIsLabelsAndDebug(MBB->begin());
else {
MachineInstr *DefMI = LIS.getInstructionFromIndex(SrcVNI->def);
assert(DefMI && "Defining instruction disappeared");
@@ -1252,6 +1252,7 @@ void InlineSpiller::insertReload(Register NewVReg,
MachineInstrSpan MIS(MI, &MBB);
TII.loadRegFromStackSlot(MBB, MI, NewVReg, StackSlot,
MRI.getRegClass(NewVReg), Register());
+ MBB.inheritBBProlog(MIS.begin(), MI);
LIS.InsertMachineInstrRangeInMaps(MIS.begin(), MI);
@@ -1297,6 +1298,7 @@ void InlineSpiller::insertSpill(Register NewVReg, bool isKill,
.addReg(NewVReg, getKillRegState(isKill));
MachineBasicBlock::iterator Spill = std::next(MI);
+ MBB.inheritBBProlog(Spill, MIS.end());
LIS.InsertMachineInstrRangeInMaps(Spill, MIS.end());
for (const MachineInstr &MI : make_range(Spill, MIS.end()))
getVDefInterval(MI, LIS);
diff --git a/llvm/lib/CodeGen/LiveRangeEdit.cpp b/llvm/lib/CodeGen/LiveRangeEdit.cpp
index 1d6b835f164680..f1c24d4e942551 100644
--- a/llvm/lib/CodeGen/LiveRangeEdit.cpp
+++ b/llvm/lib/CodeGen/LiveRangeEdit.cpp
@@ -91,6 +91,8 @@ SlotIndex LiveRangeEdit::rematerializeAt(
// to false anyway in case the isDead flag of RM.OrigMI's dest register
// is true.
(*--MI).clearRegisterDeads(DestReg);
+ MI->clearFlag(MachineInstr::BBProlog);
+ MBB.inheritBBProlog(MI, std::next(MI));
Rematted.insert(RM.ParentVNI);
++NumReMaterialization;
diff --git a/llvm/lib/CodeGen/MIRParser/MILexer.cpp b/llvm/lib/CodeGen/MIRParser/MILexer.cpp
index 0851c42c7777f1..b4cb50cbabe7c0 100644
--- a/llvm/lib/CodeGen/MIRParser/MILexer.cpp
+++ b/llvm/lib/CodeGen/MIRParser/MILexer.cpp
@@ -303,6 +303,7 @@ static MIToken::TokenKind getIdentifierKind(StringRef Identifier) {
.Case("noconvergent", MIToken::kw_noconvergent)
.Case("mmra", MIToken::kw_mmra)
.Case("lr-split", MIToken::kw_lr_split)
+ .Case("bb-prolog", MIToken::kw_bb_prolog)
.Default(MIToken::Identifier);
}
diff --git a/llvm/lib/CodeGen/MIRParser/MILexer.h b/llvm/lib/CodeGen/MIRParser/MILexer.h
index 0e28ee3635a878..eb9b728f9ed99f 100644
--- a/llvm/lib/CodeGen/MIRParser/MILexer.h
+++ b/llvm/lib/CodeGen/MIRParser/MILexer.h
@@ -155,6 +155,7 @@ struct MIToken {
kw_noconvergent,
kw_mmra,
kw_lr_split,
+ kw_bb_prolog,
// Metadata types.
kw_distinct,
diff --git a/llvm/lib/CodeGen/MIRParser/MIParser.cpp b/llvm/lib/CodeGen/MIRParser/MIParser.cpp
index 7b45d028efdd77..9440227683c4b7 100644
--- a/llvm/lib/CodeGen/MIRParser/MIParser.cpp
+++ b/llvm/lib/CodeGen/MIRParser/MIParser.cpp
@@ -1390,7 +1390,8 @@ bool MIParser::parseInstruction(unsigned &OpCode, unsigned &Flags) {
Token.is(MIToken::kw_samesign) ||
Token.is(MIToken::kw_inbounds) ||
Token.is(MIToken::kw_nonnull) ||
- Token.is(MIToken::kw_lr_split)) {
+ Token.is(MIToken::kw_lr_split) ||
+ Token.is(MIToken::kw_bb_prolog)) {
// clang-format on
// Mine frame and fast math flags
if (Token.is(MIToken::kw_frame_setup))
@@ -1437,6 +1438,8 @@ bool MIParser::parseInstruction(unsigned &OpCode, unsigned &Flags) {
Flags |= MachineInstr::NonNull;
if (Token.is(MIToken::kw_lr_split))
Flags |= MachineInstr::LRSplit;
+ if (Token.is(MIToken::kw_bb_prolog))
+ Flags |= MachineInstr::BBProlog;
lex();
}
diff --git a/llvm/lib/CodeGen/MIRPrinter.cpp b/llvm/lib/CodeGen/MIRPrinter.cpp
index 5a4aa4910c0d3d..e6359b89727c45 100644
--- a/llvm/lib/CodeGen/MIRPrinter.cpp
+++ b/llvm/lib/CodeGen/MIRPrinter.cpp
@@ -896,6 +896,8 @@ static void printMI(raw_ostream &OS, MFPrintState &State,
OS << "lr-split ";
if (MI.getFlag(MachineInstr::NonNull))
OS << "nonnull ";
+ if (MI.getFlag(MachineInstr::BBProlog))
+ OS << "bb-prolog ";
// NOTE: Please add new MIFlags also to the MI_FLAGS_STR in
// llvm/utils/UpdateTestChecks/mir.py.
diff --git a/llvm/lib/CodeGen/MachineBasicBlock.cpp b/llvm/lib/CodeGen/MachineBasicBlock.cpp
index 39c286f7d35104..ab7eeb4c7091bc 100644
--- a/llvm/lib/CodeGen/MachineBasicBlock.cpp
+++ b/llvm/lib/CodeGen/MachineBasicBlock.cpp
@@ -210,13 +210,22 @@ MachineBasicBlock::iterator MachineBasicBlock::getFirstNonPHI() {
return I;
}
+void MachineBasicBlock::inheritBBProlog(iterator Begin, iterator End) {
+ iterator Next = End;
+ while (Next != end() && (Next->isDebugOrPseudoInstr() || Next->isPosition()))
+ ++Next;
+ if (Next == end() || !Next->getFlag(MachineInstr::BBProlog))
+ return;
+ for (MachineInstr &MI : make_range(Begin, End))
+ if (!MI.isDebugOrPseudoInstr() && !MI.isPosition())
+ MI.setFlag(MachineInstr::BBProlog);
+}
+
MachineBasicBlock::iterator
MachineBasicBlock::SkipPHIsAndLabels(MachineBasicBlock::iterator I) {
- const TargetInstrInfo *TII = getParent()->getSubtarget().getInstrInfo();
-
iterator E = end();
- while (I != E && (I->isPHI() || I->isPosition() ||
- TII->isBasicBlockPrologue(*I)))
+ while (I != E &&
+ (I->isPHI() || I->isPosition() || I->getFlag(MachineInstr::BBProlog)))
++I;
// FIXME: This needs to change if we wish to bundle labels
// inside the bundle.
@@ -227,13 +236,11 @@ MachineBasicBlock::SkipPHIsAndLabels(MachineBasicBlock::iterator I) {
MachineBasicBlock::iterator
MachineBasicBlock::SkipPHIsLabelsAndDebug(MachineBasicBlock::iterator I,
- Register Reg, bool SkipPseudoOp) {
- const TargetInstrInfo *TII = getParent()->getSubtarget().getInstrInfo();
-
+ bool SkipPseudoOp) {
iterator E = end();
while (I != E && (I->isPHI() || I->isPosition() || I->isDebugInstr() ||
(SkipPseudoOp && I->isPseudoProbe()) ||
- TII->isBasicBlockPrologue(*I, Reg)))
+ I->getFlag(MachineInstr::BBProlog)))
++I;
// FIXME: This needs to change if we wish to bundle labels / dbg_values
// inside the bundle.
diff --git a/llvm/lib/CodeGen/MachineInstr.cpp b/llvm/lib/CodeGen/MachineInstr.cpp
index 4978967c7cb112..ce7dac9fcbe91a 100644
--- a/llvm/lib/CodeGen/MachineInstr.cpp
+++ b/llvm/lib/CodeGen/MachineInstr.cpp
@@ -1920,6 +1920,8 @@ void MachineInstr::print(raw_ostream &OS, ModuleSlotTracker &MST,
OS << "lr-split ";
if (getFlag(MachineInstr::NonNull))
OS << "nonnull ";
+ if (getFlag(MachineInstr::BBProlog))
+ OS << "bb-prolog ";
// Print the opcode name.
if (TII)
diff --git a/llvm/lib/CodeGen/MachineSink.cpp b/llvm/lib/CodeGen/MachineSink.cpp
index cbd2598a47210d..834effea43e014 100644
--- a/llvm/lib/CodeGen/MachineSink.cpp
+++ b/llvm/lib/CodeGen/MachineSink.cpp
@@ -343,7 +343,7 @@ static bool blockPrologueInterferes(const MachineBasicBlock *BB,
for (MachineBasicBlock::const_iterator PI = BB->getFirstNonPHI(); PI != End;
++PI) {
// Only check target defined prologue instructions
- if (!TII->isBasicBlockPrologue(*PI))
+ if (!PI->getFlag(MachineInstr::BBProlog))
continue;
for (auto &MO : MI.operands()) {
if (!MO.isReg())
@@ -1776,7 +1776,7 @@ bool MachineSinking::aggressivelySinkIntoCycle(
continue;
}
// We cannot sink before the prologue
- if (MI->isPosition() || TII->isBasicBlockPrologue(*MI)) {
+ if (MI->isPosition() || MI->getFlag(MachineInstr::BBProlog)) {
LLVM_DEBUG(dbgs() << "AggressiveCycleSink: Use is BasicBlock prologue, "
"can't sink.\n");
continue;
diff --git a/llvm/lib/CodeGen/MachineVerifier.cpp b/llvm/lib/CodeGen/MachineVerifier.cpp
index cae5012b41c331..9cecf0709a8496 100644
--- a/llvm/lib/CodeGen/MachineVerifier.cpp
+++ b/llvm/lib/CodeGen/MachineVerifier.cpp
@@ -159,6 +159,7 @@ struct MachineVerifier {
const MachineInstr *FirstNonPHI = nullptr;
const MachineInstr *FirstTerminator = nullptr;
+ const MachineInstr *FirstNonBBProlog = nullptr;
BlockSet FunctionBlocks;
BitVector regsReserved;
@@ -742,6 +743,7 @@ void
MachineVerifier::visitMachineBasicBlockBefore(const MachineBasicBlock *MBB) {
FirstTerminator = nullptr;
FirstNonPHI = nullptr;
+ FirstNonBBProlog = nullptr;
if (MRI->tracksLiveness() && hasPHIs(*MF)) {
// If this block has allocatable physical registers live-in, check that
@@ -2359,6 +2361,20 @@ void MachineVerifier::visitMachineInstrBefore(const MachineInstr *MI) {
} else if (FirstNonPHI == nullptr)
FirstNonPHI = MI;
+ // Until register allocation is done, block prolog instructions form a block
+ // prefix. Only PHIs, labels, debug instructions and pseudo probes may precede
+ // them.
+ if (!MF->getProperties().hasNoVRegs() && !MI->isPHI() && !MI->isPosition() &&
+ !MI->isDebugOrPseudoInstr()) {
+ if (!MI->getFlag(MachineInstr::BBProlog)) {
+ if (!FirstNonBBProlog)
+ FirstNonBBProlog = MI;
+ } else if (FirstNonBBProlog) {
+ report("Block prolog instruction after non-prolog instruction", MI);
+ OS << "First non-prolog instruction: " << *FirstNonBBProlog;
+ }
+ }
+
// Check the tied operands.
if (MI->isInlineAsm())
verifyInlineAsm(MI);
diff --git a/llvm/lib/CodeGen/PHIElimination.cpp b/llvm/lib/CodeGen/PHIElimination.cpp
index 054e24bc020692..57e56ab91dc892 100644
--- a/llvm/lib/CodeGen/PHIElimination.cpp
+++ b/llvm/lib/CodeGen/PHIElimination.cpp
@@ -423,6 +423,8 @@ void PHIEliminationImpl::LowerPHINode(MachineBasicBlock &MBB,
PHICopy = TII->createPHIDestinationCopy(
MBB, AfterPHIsIt, MPhi->getDebugLoc(), IncomingReg, DestReg);
}
+ MBB.inheritBBProlog(PHICopy->getIterator(),
+ std::next(PHICopy->getIterator()));
if (MPhi->peekDebugInstrNum() && IncomingReg) {
// If referred to by debug-info, store where this PHI was.
diff --git a/llvm/lib/CodeGen/RegAllocFast.cpp b/llvm/lib/CodeGen/RegAllocFast.cpp
index 524ed5e0699412..b0d2dbcc90bbe7 100644
--- a/llvm/lib/CodeGen/RegAllocFast.cpp
+++ b/llvm/lib/CodeGen/RegAllocFast.cpp
@@ -616,7 +616,9 @@ void RegAllocFastImpl::spill(MachineBasicBlock::iterator Before,
LLVM_DEBUG(dbgs() << " to stack slot #" << FI << '\n');
const TargetRegisterClass &RC = *MRI->getRegClass(VirtReg);
+ MachineInstrSpan MIS(Before, MBB);
TII->storeRegToStackSlot(*MBB, Before, AssignedReg, Kill, FI, &RC, VirtReg);
+ MBB->inheritBBProlog(MIS.begin(), Before);
++NumStores;
MachineBasicBlock::iterator FirstTerm = MBB->getFirstTerminator();
@@ -673,7 +675,9 @@ void RegAllocFastImpl::reload(MachineBasicBlock::iterator Before,
<< printReg(PhysReg, TRI) << '\n');
int FI = getStackSpaceFor(VirtReg);
const TargetRegisterClass &RC = *MRI->getRegClass(VirtReg);
+ MachineInstrSpan MIS(Before, MBB);
TII->loadRegFromStackSlot(*MBB, Before, PhysReg, FI, &RC, VirtReg);
+ MBB->inheritBBProlog(MIS.begin(), Before);
++NumLoads;
}
@@ -691,7 +695,7 @@ MachineBasicBlock::iterator RegAllocFastImpl::getMBBBeginInsertionPoint(
}
// Skip prologues and inlineasm_br spills to place reloads afterwards.
- if (!TII->isBasicBlockPrologue(*I) && !mayBeSpillFromInlineAsmBr(*I))
+ if (!I->getFlag(MachineInstr::BBProlog) && !mayBeSpillFromInlineAsmBr(*I))
break;
// However if a prolog instruction reads a register that needs to be
diff --git a/llvm/lib/CodeGen/RegAllocGreedy.cpp b/llvm/lib/CodeGen/RegAllocGreedy.cpp
index d534a71a38d0d7..4a6f8f5831e978 100644
--- a/llvm/lib/CodeGen/RegAllocGreedy.cpp
+++ b/llvm/lib/CodeGen/RegAllocGreedy.cpp
@@ -841,8 +841,7 @@ bool RAGreedy::addThroughConstraints(InterferenceCache::Cursor Intf,
return false;
// Interference for the live-in value.
- Register Reg = SA->getParent().reg();
- auto InsertPt = MBB->SkipPHIsLabelsAndDebug(MBB->begin(), Reg);
+ auto InsertPt = MBB->SkipPHIsLabelsAndDebug(MBB->begin());
SlotIndex InsertIdx = InsertPt == MBB->end()
? Indexes->getMBBEndIdx(MBB)
: LIS->getInstructionIndex(*InsertPt);
diff --git a/llvm/lib/CodeGen/SplitKit.cpp b/llvm/lib/CodeGen/SplitKit.cpp
index 404cfe43289d6e..76fe405e7116c3 100644
--- a/llvm/lib/CodeGen/SplitKit.cpp
+++ b/llvm/lib/CodeGen/SplitKit.cpp
@@ -672,6 +672,7 @@ VNInfo *SplitEditor::defFromParent(unsigned RegIdx, const VNInfo *ParentVNI,
}
SlotIndex Def;
+ MachineInstrSpan MIS(I, &MBB);
if (LaneMask.none()) {
const MCInstrDesc &Desc = TII.get(TargetOpcode::IMPLICIT_DEF);
MachineInstr *ImplicitDef = BuildMI(MBB, I, DebugLoc(), Desc, Reg);
@@ -681,6 +682,7 @@ VNInfo *SplitEditor::defFromParent(unsigned RegIdx, const VNInfo *ParentVNI,
++NumCopies;
Def = buildCopy(Edit->getReg(), Reg, LaneMask, MBB, I, Late, RegIdx);
}
+ MBB.inheritBBProlog(MIS.begin(), I);
// Define the value in Reg.
return defValue(RegIdx, ParentVNI, Def, false);
@@ -851,9 +853,8 @@ SlotIndex SplitEditor::leaveIntvAtTop(MachineBasicBlock &MBB) {
}
unsigned RegIdx = 0;
- Register Reg = LIS.getInterval(Edit->get(RegIdx)).reg();
VNInfo *VNI = defFromParent(RegIdx, ParentVNI, Start, MBB,
- MBB.SkipPHIsLabelsAndDebug(MBB.begin(), Reg));
+ MBB.SkipPHIsLabelsAndDebug(MBB.begin()));
RegAssign.insert(Start, VNI->def, OpenIdx);
LLVM_DEBUG(dump());
return VNI->def;
diff --git a/llvm/lib/CodeGen/TargetInstrInfo.cpp b/llvm/lib/CodeGen/TargetInstrInfo.cpp
index 92fc628e888e5e..0a0d9261036b87 100644
--- a/llvm/lib/CodeGen/TargetInstrInfo.cpp
+++ b/llvm/lib/CodeGen/TargetInstrInfo.cpp
@@ -788,6 +788,10 @@ MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
// call instructions. We need copy it form old instruction.
NewMI->cloneInstrSymbols(MF, MI);
+ // NewMI replaces MI.
+ if (MI.getFlag(MachineInstr::BBProlog))
+ NewMI->setFlag(MachineInstr::BBProlog);
+
return NewMI;
}
@@ -801,6 +805,7 @@ MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
const MachineOperand &MO = MI.getOperand(1 - Ops[0]);
MachineBasicBlock::iterator Pos = MI;
+ MachineInstrSpan MIS(Pos, MBB);
if (Flags == MachineMemOperand::MOStore) {
if (MO.isUndef()) {
// If this is an undef copy, we do not need to bother we inserting spill
@@ -812,6 +817,7 @@ MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
}
} else
loadRegFromStackSlot(*MBB, Pos, MO.getReg(), FI, RC, Register());
+ MBB->inheritBBProlog(MIS.begin(), Pos);
return &*--Pos;
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp b/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp
index 140cf58e8fdd3a..20b0968e1f8400 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp
@@ -1818,8 +1818,11 @@ bool AMDGPUInstructionSelector::selectEndCfIntrinsic(MachineInstr &MI) const {
// FIXME: Manually selecting to avoid dealing with the SReg_1 trick
// SelectionDAG uses for wave32 vs wave64.
MachineBasicBlock *BB = MI.getParent();
- BuildMI(*BB, &MI, MI.getDebugLoc(), TII.get(AMDGPU::SI_END_CF))
- .add(MI.getOperand(1));
+ MachineInstr *EndCf =
+ BuildMI(*BB, &MI, MI.getDebugLoc(), TII.get(AMDGPU::SI_END_CF))
+ .add(MI.getOperand(1));
+ // Exec restore at the top of the join block.
+ SIInstrInfo::setBBPrologIfAtBlockStart(*EndCf);
Register Reg = MI.getOperand(1).getReg();
MI.eraseFromParent();
diff --git a/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp b/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp
index 964ddde0fbb017..0f9be4c20b591b 100644
--- a/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp
+++ b/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp
@@ -443,10 +443,9 @@ static bool isReachable(const MachineInstr *From,
}
// Return the first non-prologue instruction in the block.
-static MachineBasicBlock::iterator
-getFirstNonPrologue(MachineBasicBlock *MBB, const TargetInstrInfo *TII) {
+static MachineBasicBlock::iterator getFirstNonPrologue(MachineBasicBlock *MBB) {
MachineBasicBlock::iterator I = MBB->getFirstNonPHI();
- while (I != MBB->end() && TII->isBasicBlockPrologue(*I))
+ while (I != MBB->end() && I->getFlag(MachineInstr::BBProlog))
++I;
return I;
@@ -558,7 +557,7 @@ static bool hoistAndMergeSGPRInits(unsigned Reg,
continue;
}
- MachineBasicBlock::iterator I = getFirstNonPrologue(MBB, TII);
+ MachineBasicBlock::iterator I = getFirstNonPrologue(MBB);
if (!interferes(MI1, I) && !interferes(MI2, I)) {
LLVM_DEBUG(dbgs()
<< "Erasing from "
@@ -597,11 +596,11 @@ static bool hoistAndMergeSGPRInits(unsigned Reg,
auto &Defs = Init.second;
for (auto *MI : Defs) {
auto *MBB = MI->getParent();
- MachineInstr &BoundaryMI = *getFirstNonPrologue(MBB, TII);
+ MachineInstr &BoundaryMI = *getFirstNonPrologue(MBB);
MachineBasicBlock::reverse_iterator B(BoundaryMI);
// Check if B should actually be a boundary. If not set the previous
// instruction as the boundary instead.
- if (!TII->isBasicBlockPrologue(*B))
+ if (!B->getFlag(MachineInstr::BBProlog))
B++;
auto R = std::next(MI->getReverseIterator());
diff --git a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
index f718d9e4032090..6be53eb538e95c 100644
--- a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
@@ -19807,6 +19807,12 @@ void SITargetLowering::AdjustInstrPostInstrSelection(MachineInstr &MI,
MachineFunction *MF = MI.getMF();
MachineRegisterInfo &MRI = MF->getRegInfo();
+ if (MI.getOpcode() == AMDGPU::SI_END_CF) {
+ // Exec restore at the top of the join block.
+ SIInstrInfo::setBBPrologIfAtBlockStart(MI);
+ return;
+ }
+
if (TII->isVOP3(MI.getOpcode())) {
// Make sure constant bus requirements are respected.
TII->legalizeOperandsVOP3(MRI, MI);
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
index ea16b6e886a44f..04bc3018f3ed36 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
@@ -10452,51 +10452,6 @@ unsigned SIInstrInfo::getLiveRangeSplitOpcode(Register SrcReg,
return AMDGPU::COPY;
}
-bool SIInstrInfo::canAddToBBProlog(const MachineInstr &MI) const {
- uint32_t Opcode = MI.getOpcode();
- // Check if it is SGPR spill or wwm-register spill Opcode.
- if (isSGPRSpill(Opcode) || isWWMRegSpillOpcode(Opcode))
- return true;
-
- const MachineFunction *MF = MI.getMF();
- const MachineRegisterInfo &MRI = MF->getRegInfo();
- const SIMachineFunctionInfo *MFI = MF->getInfo<SIMachineFunctionInfo>();
-
- // See if this is Liverange split instruction inserted for SGPR or
- // wwm-register. The implicit def inserted for wwm-registers should also be
- // included as they can appear at the bb begin.
- bool IsLRSplitInst = MI.getFlag(MachineInstr::LRSplit);
- if (!IsLRSplitInst && Opcode != AMDGPU::IMPLICIT_DEF)
- return false;
-
- Register Reg = MI.getOperand(0).getReg();
- if (RI.isSGPRClass(RI.getRegClassForReg(MRI, Reg)))
- return IsLRSplitInst;
-
- return MFI->isWWMReg(Reg);
-}
-
-bool SIInstrInfo::isBasicBlockPrologue(const MachineInstr &MI,
- Register Reg) const {
- // We need to handle instructions which may be inserted during register
- // allocation to handle the prolog. The initial prolog instruction may have
- // been separated from the start of the block by spills and copies inserted
- // needed by the prolog. However, the insertions for scalar registers can
- // always be placed at the BB top as they are independent of the exec mask
- // value.
- bool IsNullOrVectorRegister = true;
- if (Reg) {
- const MachineFunction *MF = MI.getMF();
- const MachineRegisterInfo &MRI = MF->getRegInfo();
- IsNullOrVectorRegister = !RI.isSGPRClass(RI.getRegClassForReg(MRI, Reg));
- }
-
- return IsNullOrVectorRegister &&
- (canAddToBBProlog(MI) ||
- (!MI.isTerminator() && MI.getOpcode() != AMDGPU::COPY &&
- MI.modifiesRegister(AMDGPU::EXEC, &RI)));
-}
-
MachineInstrBuilder
SIInstrInfo::getAddNoCarry(MachineBasicBlock &MBB,
MachineBasicBlock::iterator I,
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.h b/llvm/lib/Target/AMDGPU/SIInstrInfo.h
index 48eba6b2a567df..fa3cb9a8a2cc11 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.h
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.h
@@ -924,6 +924,14 @@ class SIInstrInfo final : public AMDGPUGenInstrInfo {
static bool isSpill(const MachineInstr &MI) { return isSpill(MI.getDesc()); }
+ /// Mark \p MI as a block prolog instruction if it is at the block start.
+ static void setBBPrologIfAtBlockStart(MachineInstr &MI) {
+ MachineBasicBlock &MBB = *MI.getParent();
+ if (!MI.isTerminator() &&
+ MBB.SkipPHIsLabelsAndDebug(MBB.begin()) == MI.getIterator())
+ MI.setFlag(MachineInstr::BBProlog);
+ }
+
static bool isWWMRegSpillOpcode(uint32_t Opcode) {
return Opcode == AMDGPU::SI_SPILL_WWM_V32_SAVE ||
Opcode == AMDGPU::SI_SPILL_WWM_AV32_SAVE ||
@@ -1684,11 +1692,6 @@ class SIInstrInfo final : public AMDGPUGenInstrInfo {
unsigned getLiveRangeSplitOpcode(Register Reg,
const MachineFunction &MF) const override;
- bool isBasicBlockPrologue(const MachineInstr &MI,
- Register Reg = Register()) const override;
-
- bool canAddToBBProlog(const MachineInstr &MI) const;
-
MachineInstr *createPHIDestinationCopy(MachineBasicBlock &MBB,
MachineBasicBlock::iterator InsPt,
const DebugLoc &DL, Register Src,
diff --git a/llvm/lib/Target/AMDGPU/SIInstructions.td b/llvm/lib/Target/AMDGPU/SIInstructions.td
index eaece88e985251..0682d70f40a12a 100644
--- a/llvm/lib/Target/AMDGPU/SIInstructions.td
+++ b/llvm/lib/Target/AMDGPU/SIInstructions.td
@@ -630,6 +630,7 @@ def SI_LOOP : CFPseudoInstSI <
def SI_END_CF : CFPseudoInstSI <
(outs), (ins SReg_1:$saved), [], 1, 1> {
let Size = 4;
+ let hasPostISelHook = 1; // Sets MachineInstr::BBProlog.
let isAsCheapAsAMove = 1;
let isReMaterializable = 1;
let hasSideEffects = 1;
diff --git a/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp b/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp
index 6c87ba5438f3d0..55274b8e3cdb76 100644
--- a/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp
+++ b/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp
@@ -321,6 +321,7 @@ void SILowerControlFlow::emitElse(MachineInstr &MI) {
MachineInstr *OrSaveExec =
BuildMI(MBB, Start, DL, TII->get(LMC.OrSaveExecOpc), SaveReg)
.add(MI.getOperand(1)); // Saved EXEC
+ OrSaveExec->setFlag(MachineInstr::BBProlog);
if (LV)
LV->replaceKillInstruction(SrcReg, MI, *OrSaveExec);
@@ -518,6 +519,10 @@ MachineBasicBlock *SILowerControlFlow::emitEndCf(MachineInstr &MI) {
MachineInstr *NewMI = BuildMI(MBB, InsPt, DL, TII->get(Opcode), LMC.ExecReg)
.addReg(LMC.ExecReg)
.add(MI.getOperand(0));
+ if (NeedBlockSplit)
+ NewMI->clearFlag(MachineInstr::BBProlog);
+ else
+ NewMI->setFlag(MachineInstr::BBProlog);
if (LV) {
LV->replaceKillInstruction(DataReg, MI, *NewMI);
diff --git a/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp b/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp
index d0a57e163d29a0..fd7809e1fb76ef 100644
--- a/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp
+++ b/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp
@@ -132,6 +132,7 @@ static void insertCSRSaves(const GCNSubtarget &ST, MachineBasicBlock &SaveBlock,
bool Success = TFI->spillCalleeSavedRegisters(SaveBlock, I, CSI, TRI);
assert(Success && "spillCalleeSavedRegisters should always succeed");
(void)Success;
+ SaveBlock.inheritBBProlog(MIS.begin(), I);
// TFI doesn't update Indexes and LIS, so we have to do it separately.
if (Indexes)
@@ -581,6 +582,7 @@ bool SILowerSGPRSpills::run(MachineFunction &MF) {
MachineBasicBlock &Block = *IP.MBB;
DebugLoc DL = Block.findDebugLoc(IP.It);
auto MIB = BuildMI(Block, IP.It, DL, TII->get(AMDGPU::IMPLICIT_DEF), Reg);
+ Block.inheritBBProlog(MIB->getIterator(), IP.It);
// Add WWM flag to the virtual register.
FuncInfo->setFlag(Reg, AMDGPU::VirtRegFlag::WWM_REG);
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
index bf5c821611dd8e..877772c7732893 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
@@ -2166,6 +2166,7 @@ bool SIRegisterInfo::spillSGPR(MachineBasicBlock::iterator MI, int Index,
"undef spill should have been deleted earlier");
SGPRSpillBuilder SB(*this, *ST.getInstrInfo(), isWave32, MI, Index, RS);
+ MachineInstrSpan MIS(MI, MI->getParent());
ArrayRef<SpilledReg> VGPRSpills =
SpillToPhysVGPRLane ? SB.MFI.getSGPRSpillToPhysicalVGPRLanes(Index)
@@ -2310,6 +2311,8 @@ bool SIRegisterInfo::spillSGPR(MachineBasicBlock::iterator MI, int Index,
SB.restore();
}
+ // The expansion replaces MI.
+ MI->getParent()->inheritBBProlog(MIS.begin(), MI);
MI->eraseFromParent();
SB.MFI.addToSpilledSGPRs(SB.NumSubRegs);
@@ -2324,6 +2327,7 @@ bool SIRegisterInfo::restoreSGPR(MachineBasicBlock::iterator MI, int Index,
LiveIntervals *LIS, bool OnlyToVGPR,
bool SpillToPhysVGPRLane) const {
SGPRSpillBuilder SB(*this, *ST.getInstrInfo(), isWave32, MI, Index, RS);
+ MachineInstrSpan MIS(MI, MI->getParent());
ArrayRef<SpilledReg> VGPRSpills =
SpillToPhysVGPRLane ? SB.MFI.getSGPRSpillToPhysicalVGPRLanes(Index)
@@ -2391,6 +2395,8 @@ bool SIRegisterInfo::restoreSGPR(MachineBasicBlock::iterator MI, int Index,
SB.restore();
}
+ // The expansion replaces MI.
+ MI->getParent()->inheritBBProlog(MIS.begin(), MI);
MI->eraseFromParent();
if (LIS)
diff --git a/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp b/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp
index 082fdf645ca261..0cc6112b0210f0 100644
--- a/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp
+++ b/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp
@@ -1228,6 +1228,7 @@ void SIWholeQuadMode::toExact(MachineBasicBlock &MBB,
.addReg(LiveMaskReg);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
LIS->removeAllRegUnitsForPhysReg(AMDGPU::EXEC);
StateTransition[MI] = StateExact;
@@ -1247,6 +1248,7 @@ void SIWholeQuadMode::toWQM(MachineBasicBlock &MBB,
.addReg(LMC.ExecReg);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
StateTransition[MI] = StateWQM;
}
@@ -1268,6 +1270,7 @@ void SIWholeQuadMode::toStrictMode(MachineBasicBlock &MBB,
MI = BuildMI(MBB, Before, DL, TII->get(AMDGPU::ENTER_STRICT_WQM), SaveOrig)
.addImm(-1);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
StateTransition[MI] = StrictStateNeeded;
}
@@ -1293,6 +1296,7 @@ void SIWholeQuadMode::fromStrictMode(MachineBasicBlock &MBB,
BuildMI(MBB, Before, DL, TII->get(AMDGPU::EXIT_STRICT_WQM), LMC.ExecReg)
.addReg(SavedOrig);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
StateTransition[MI] = NonStrictState;
}
@@ -1612,7 +1616,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
Register EntryExec = MRI->createVirtualRegister(TRI->getBoolRC());
MachineInstr *SaveExec = BuildMI(*MBB, MBB->begin(), MI.getDebugLoc(),
TII->get(LMC.OrSaveExecOpc), EntryExec)
- .addImm(-1);
+ .addImm(-1)
+ .setMIFlag(MachineInstr::BBProlog);
// Replace all uses of MI's destination reg with EntryExec.
MRI->replaceRegWith(MI.getOperand(0).getReg(), EntryExec);
@@ -1634,7 +1639,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
// This should be before all vector instructions.
MachineInstr *InitMI = BuildMI(*MBB, MBB->begin(), MI.getDebugLoc(),
TII->get(LMC.MovOpc), LMC.ExecReg)
- .addImm(MI.getOperand(0).getImm());
+ .addImm(MI.getOperand(0).getImm())
+ .setMIFlag(MachineInstr::BBProlog);
if (LIS) {
LIS->RemoveMachineInstrFromMaps(MI);
LIS->InsertMachineInstrInMaps(*InitMI);
@@ -1667,6 +1673,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
// If first instruction is definition then move pointer after it.
FirstMI = &*std::next(FirstMI->getIterator());
}
+ // The input copy feeds the exec setup below.
+ DefInstr->setFlag(MachineInstr::BBProlog);
}
}
@@ -1686,6 +1694,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
.addImm(WavefrontSize);
auto CmovMI =
BuildMI(*MBB, FirstMI, DL, TII->get(LMC.CMovOpc), LMC.ExecReg).addImm(-1);
+ for (MachineInstr *I : {&*BfeMI, &*BfmMI, &*CmpMI, &*CmovMI})
+ I->setFlag(MachineInstr::BBProlog);
if (!LIS) {
MI.eraseFromParent();
@@ -1758,6 +1768,7 @@ bool SIWholeQuadMode::run(MachineFunction &MF) {
MachineInstr *MI =
BuildMI(Entry, EntryMI, DebugLoc(), TII->get(AMDGPU::COPY), LiveMaskReg)
.addReg(LMC.ExecReg);
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
Changed = true;
}
@@ -1791,6 +1802,7 @@ bool SIWholeQuadMode::run(MachineFunction &MF) {
auto MI =
BuildMI(Entry, EntryMI, DebugLoc(), TII->get(LMC.WQMOpc), LMC.ExecReg)
.addReg(LMC.ExecReg);
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
lowerKillInstrs(true);
Changed = true;
diff --git a/llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir b/llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir
new file mode 100644
index 00000000000000..e76379cf98f54c
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir
@@ -0,0 +1,82 @@
+# RUN: llc -mtriple=amdgpu9.42-amd-amdhsa -start-before=greedy,2 -stop-after=greedy,2 -stress-regalloc=3 -o - %s | FileCheck %s
+
+# The sgpr-regalloc pipeline rematerialized an SMEM load into the Flow block,
+# ahead of the exec restore of the if/else region. The exec restore carries the
+# bb-prolog flag, so the vgpr-regalloc pipeline must place the spill it inserts
+# in bb.3 after the S_OR_SAVEEXEC_B64. Otherwise the spill would only be
+# executed for the lanes of the first branch arm. The rematerialized load is not
+# part of the prolog: the spill goes ahead of it.
+
+# CHECK-LABEL: name: sgpr_remat_in_bb_prolog
+# CHECK: bb.3:
+# CHECK: bb-prolog S_OR_SAVEEXEC_B64
+# CHECK-NEXT: SI_SPILL_AV64_SAVE
+# CHECK-NEXT: S_LOAD_DWORDX8_IMM_ec
+# CHECK-NEXT: V_LSHL_ADD_U64_e64
+
+--- |
+ define amdgpu_kernel void @sgpr_remat_in_bb_prolog() { ret void }
+...
+---
+name: sgpr_remat_in_bb_prolog
+tracksRegLiveness: true
+machineFunctionInfo:
+ stackPtrOffsetReg: '$sgpr32'
+body: |
+ bb.0:
+ successors: %bb.1(0x40000000), %bb.2(0x40000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr64_sgpr65, $sgpr68_sgpr69, $vgpr0_vgpr1, $vgpr2_vgpr3
+
+ %0:vreg_64_align2 = COPY $vgpr0_vgpr1
+ %1:vreg_64_align2 = COPY $vgpr2_vgpr3
+ $exec = S_MOV_B64_term killed renamable $sgpr30_sgpr31
+ S_CBRANCH_EXECZ %bb.2, implicit $exec
+ S_BRANCH %bb.1
+
+ bb.1:
+ successors: %bb.2(0x80000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr34_sgpr35, $sgpr64_sgpr65, $sgpr68_sgpr69
+
+ bb.2:
+ successors: %bb.5(0x40000000), %bb.3(0x40000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr34_sgpr35, $sgpr64_sgpr65, $sgpr68_sgpr69
+
+ %2:vreg_64_align2 = COPY $sgpr68_sgpr69, implicit $exec
+ $exec = S_MOV_B64_term killed renamable $sgpr34_sgpr35
+ S_CBRANCH_EXECZ %bb.3, implicit $exec
+ S_BRANCH %bb.5
+
+ bb.3:
+ successors: %bb.4(0x40000000), %bb.6(0x40000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr64_sgpr65
+
+ renamable $sgpr28_sgpr29 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr28_sgpr29, implicit-def $exec, implicit-def $scc, implicit $exec
+ early-clobber renamable $sgpr8_sgpr9_sgpr10_sgpr11_sgpr12_sgpr13_sgpr14_sgpr15 = S_LOAD_DWORDX8_IMM_ec renamable $sgpr4_sgpr5, 32, 0 :: (dereferenceable invariant load (s256), align 16, addrspace 4)
+ %3:vreg_64_align2 = V_LSHL_ADD_U64_e64 $sgpr64_sgpr65, 0, %1, implicit $exec
+ $exec = S_XOR_B64_term $exec, renamable $sgpr28_sgpr29, implicit-def $scc
+ S_CBRANCH_EXECZ %bb.6, implicit $exec
+ S_BRANCH %bb.4
+
+ bb.4:
+ successors: %bb.6(0x80000000)
+ liveins: $sgpr8_sgpr9_sgpr10_sgpr11_sgpr12_sgpr13_sgpr14_sgpr15
+
+ S_BRANCH %bb.6
+
+ bb.5:
+ successors: %bb.3(0x80000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr64_sgpr65
+
+ %2:vreg_64_align2 = COPY $sgpr64_sgpr65, implicit $exec
+ S_BRANCH %bb.3
+
+ bb.6:
+ liveins: $sgpr8_sgpr9_sgpr10_sgpr11_sgpr12_sgpr13_sgpr14_sgpr15
+
+ %4:vreg_64_align2 = IMPLICIT_DEF
+ %5:vreg_64_align2 = IMPLICIT_DEF
+ %4:vreg_64_align2 = V_FMAC_F64_e32 %4, %5, %4, implicit $mode, implicit $exec
+ %6:vreg_64_align2 = V_FMA_F64_e64 1, %3, 0, %4, 0, 4607182418800017408, 0, 0, implicit $mode, implicit $exec
+ %7:vreg_64_align2 = V_LSHL_ADD_U64_e64 %2, 0, %0, implicit $exec
+ S_ENDPGM 0, implicit %6, implicit %7
+...
diff --git a/llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir b/llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir
new file mode 100644
index 00000000000000..10aad7514eb424
--- /dev/null
+++ b/llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir
@@ -0,0 +1,17 @@
+# RUN: llc -mtriple=amdgpu9.00 -run-pass=none -o - %s | FileCheck %s
+#
+# Round-trip test for the 'bb-prolog' MIR text spelling of
+# MachineInstr::BBProlog.
+
+---
+name: bb_prolog_roundtrip
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $sgpr0_sgpr1
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ S_ENDPGM 0, implicit %0
+...
+
+# CHECK-LABEL: name: bb_prolog_roundtrip
+# CHECK: %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr0_sgpr1
diff --git a/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir b/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir
index 6a517b7101c6e9..933f45fae6b66f 100644
--- a/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir
+++ b/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir
@@ -2,8 +2,7 @@
#
# Round-trip test for the 'lr-split' MIR text spelling of
# MachineInstr::LRSplit. The flag is set by SplitKit on copies inserted for
-# live-range splitting; AMDGPU's SIInstrInfo::canAddToBBProlog uses it to
-# recognize WWM_COPY / IMPLICIT_DEF instructions as part of the BB prologue.
+# live-range splitting.
---
name: lr_split_roundtrip
diff --git a/llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir b/llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir
new file mode 100644
index 00000000000000..245c257643324d
--- /dev/null
+++ b/llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir
@@ -0,0 +1,43 @@
+# RUN: not --crash llc -mtriple=amdgpu9.00 -verify-machineinstrs -run-pass=none -filetype=null %s 2>&1 | FileCheck %s
+
+# Block prolog instructions must form a block prefix, only preceded by PHIs,
+# labels, debug instructions and pseudo probes.
+
+# CHECK-NOT: Bad machine code
+# CHECK: *** Bad machine code: Block prolog instruction after non-prolog instruction ***
+# CHECK-NEXT: - function: bad
+# CHECK-NEXT: - basic block: %bb.0
+# CHECK-NEXT: - instruction: %1:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64
+# CHECK-NEXT: First non-prolog instruction: %0:sreg_64 = COPY $sgpr0_sgpr1
+# CHECK-NOT: Bad machine code
+
+---
+name: good
+tracksRegLiveness: true
+body: |
+ bb.0:
+ successors: %bb.1
+ liveins: $sgpr0_sgpr1
+ %0:sreg_64 = COPY $sgpr0_sgpr1
+ S_BRANCH %bb.1
+
+ bb.1:
+ %1:sreg_64 = PHI %0, %bb.0
+ DBG_VALUE
+ PSEUDO_PROBE 1, 1, 0, 0
+ %2:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %1, implicit-def $exec, implicit-def $scc, implicit $exec
+ DBG_VALUE
+ $exec = bb-prolog S_OR_B64 $exec, %2, implicit-def $scc
+ S_ENDPGM 0
+...
+
+---
+name: bad
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $sgpr0_sgpr1
+ %0:sreg_64 = COPY $sgpr0_sgpr1
+ %1:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %0, implicit-def $exec, implicit-def $scc, implicit $exec
+ S_ENDPGM 0, implicit %1
+...
diff --git a/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp b/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp
index ef0d40b8644f9d..00847acf1125c2 100644
--- a/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp
+++ b/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp
@@ -105,4 +105,105 @@ TEST(FindDebugLocTest, DifferentIterators) {
DIB.finalize();
}
+static MachineInstr *createMI(MachineFunction &MF, unsigned Opcode,
+ bool BBProlog = false) {
+ MCInstrDesc Desc = {Opcode, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
+ MachineInstr *MI = MF.CreateMachineInstr(Desc, DebugLoc());
+ if (BBProlog)
+ MI->setFlag(MachineInstr::BBProlog);
+ return MI;
+}
+
+// Instructions inserted in front of a block prolog instruction join the prolog.
+TEST(BBPrologTest, InheritBBProlog) {
+ LLVMContext Ctx;
+ Module Mod("Module", Ctx);
+ auto MF = createMachineFunction(Ctx, Mod);
+ auto &MBB = *MF->CreateMachineBasicBlock();
+
+ MachineInstr *Prolog = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Body = createMI(*MF, TargetOpcode::COPY);
+ MBB.push_back(Prolog);
+ MBB.push_back(Body);
+
+ MachineInstr *A = createMI(*MF, TargetOpcode::COPY);
+ MachineInstr *B = createMI(*MF, TargetOpcode::COPY);
+ MBB.insert(MachineBasicBlock::iterator(Prolog), A);
+ MBB.insert(MachineBasicBlock::iterator(Prolog), B);
+ MBB.inheritBBProlog(A->getIterator(), Prolog->getIterator());
+ EXPECT_TRUE(A->getFlag(MachineInstr::BBProlog));
+ EXPECT_TRUE(B->getFlag(MachineInstr::BBProlog));
+
+ // At or after the end of the prolog.
+ MachineInstr *C = createMI(*MF, TargetOpcode::COPY);
+ MBB.insert(MachineBasicBlock::iterator(Body), C);
+ MBB.inheritBBProlog(C->getIterator(), Body->getIterator());
+ EXPECT_FALSE(C->getFlag(MachineInstr::BBProlog));
+
+ MachineInstr *D = createMI(*MF, TargetOpcode::COPY);
+ MBB.push_back(D);
+ MBB.inheritBBProlog(D->getIterator(), MBB.end());
+ EXPECT_FALSE(D->getFlag(MachineInstr::BBProlog));
+}
+
+// Debug instructions, labels and pseudo probes in front of the prolog are
+// looked through, and never become prolog instructions themselves.
+TEST(BBPrologTest, InheritLooksThroughDebugLabelsAndProbes) {
+ LLVMContext Ctx;
+ Module Mod("Module", Ctx);
+ auto MF = createMachineFunction(Ctx, Mod);
+ auto &MBB = *MF->CreateMachineBasicBlock();
+
+ MachineInstr *Dbg = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *Label = createMI(*MF, TargetOpcode::EH_LABEL);
+ MachineInstr *Probe = createMI(*MF, TargetOpcode::PSEUDO_PROBE);
+ MachineInstr *Prolog = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Dbg2 = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *Body = createMI(*MF, TargetOpcode::COPY);
+ for (MachineInstr *MI : {Dbg, Label, Probe, Prolog, Dbg2, Body})
+ MBB.push_back(MI);
+
+ MachineInstr *A = createMI(*MF, TargetOpcode::COPY);
+ MachineInstr *DbgIn = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *ProbeIn = createMI(*MF, TargetOpcode::PSEUDO_PROBE);
+ MBB.insert(MachineBasicBlock::iterator(Dbg), A);
+ MBB.insert(MachineBasicBlock::iterator(Dbg), DbgIn);
+ MBB.insert(MachineBasicBlock::iterator(Dbg), ProbeIn);
+ MBB.inheritBBProlog(A->getIterator(), Dbg->getIterator());
+ EXPECT_TRUE(A->getFlag(MachineInstr::BBProlog));
+ EXPECT_FALSE(DbgIn->getFlag(MachineInstr::BBProlog));
+ EXPECT_FALSE(ProbeIn->getFlag(MachineInstr::BBProlog));
+
+ MachineInstr *B = createMI(*MF, TargetOpcode::COPY);
+ MBB.insert(MachineBasicBlock::iterator(Dbg2), B);
+ MBB.inheritBBProlog(B->getIterator(), Dbg2->getIterator());
+ EXPECT_FALSE(B->getFlag(MachineInstr::BBProlog));
+}
+
+// The block-entry walks stop after the prolog.
+TEST(BBPrologTest, SkipPHIsSkipsProlog) {
+ LLVMContext Ctx;
+ Module Mod("Module", Ctx);
+ auto MF = createMachineFunction(Ctx, Mod);
+ auto &MBB = *MF->CreateMachineBasicBlock();
+
+ MachineInstr *PHI = createMI(*MF, TargetOpcode::PHI);
+ MachineInstr *Label = createMI(*MF, TargetOpcode::EH_LABEL);
+ MachineInstr *Prolog = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Dbg = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *Prolog2 = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Body = createMI(*MF, TargetOpcode::COPY);
+ for (MachineInstr *MI : {PHI, Label, Prolog, Dbg, Prolog2, Body})
+ MBB.push_back(MI);
+
+ EXPECT_EQ(&*MBB.SkipPHIsLabelsAndDebug(MBB.begin()), Body);
+ MBB.erase(Dbg);
+ EXPECT_EQ(&*MBB.SkipPHIsAndLabels(MBB.begin()), Body);
+
+ Prolog->clearFlag(MachineInstr::BBProlog);
+ Prolog2->clearFlag(MachineInstr::BBProlog);
+ EXPECT_EQ(&*MBB.SkipPHIsLabelsAndDebug(MBB.begin()), Prolog);
+ EXPECT_EQ(&*MBB.SkipPHIsAndLabels(MBB.begin()), Prolog);
+}
+
} // end namespace
diff --git a/llvm/utils/UpdateTestChecks/mir.py b/llvm/utils/UpdateTestChecks/mir.py
index ff143177e65d9b..e57bad288183f2 100644
--- a/llvm/utils/UpdateTestChecks/mir.py
+++ b/llvm/utils/UpdateTestChecks/mir.py
@@ -21,7 +21,7 @@
MI_FLAGS_STR = (
r"(frame-setup |frame-destroy |nnan |ninf |nsz |arcp |contract |afn "
r"|reassoc |nuw |nsw |exact |nofpexcept |nomerge |unpredictable "
- r"|noconvergent |nneg |disjoint |nusw |samesign |inbounds |lr-split "
+ r"|noconvergent |nneg |disjoint |nusw |samesign |inbounds |lr-split |bb-prolog "
r"|nonnull )*"
)
VREG_DEF_FLAGS_STR = r"(?:dead |undef )*"
>From ac2607bc0868e3b1b853dd0ec5e00ab71b41546d Mon Sep 17 00:00:00 2001
From: Gabriel Baraldi <baraldigabriel at gmail.com>
Date: Tue, 22 Sep 2026 15:24:17 +0000
Subject: [PATCH 2/3] [AMDGPU] Update tests for the BBProlog flag
MIR inputs that start after control flow lowering must carry bb-prolog on
their exec restores and SI_END_CFs; without it the prolog is no longer
inferred. The flag was added mechanically following the heuristic the
removed isBasicBlockPrologue implemented: at the block prefix, a run of
RA-inserted-looking instructions and non-terminator EXEC writes up to the
last EXEC write.
Expectations are regenerated. Most differences only print the flag. The
codegen differences are reorderings at block starts: sgpr-regalloc
insertions now follow the exec restore, and -O0 VGPR reloads precede
SGPR spill reloads that used to count as prolog.
Co-Authored-By: Claude Fable 5.1 <noreply at anthropic.com>
---
.../GlobalISel/image-waterfall-loop-O0.ll | 14 +-
.../GlobalISel/llvm.amdgcn.wqm.demote.ll | 44 +-
.../AMDGPU/NextUseAnalysis/acyclic-014bb.mir | 12 +-
.../AMDGPU/NextUseAnalysis/acyclic-770bb.mir | 768 +++++++++---------
.../acyclic-cfg-with-self-loop.mir | 24 +-
.../acyclic-phi-merge-distances.mir | 6 +-
.../complex-acyclic-cfg-with-4-self-loops.mir | 30 +-
.../complex-control-flow-11blocks.mir | 24 +-
.../complex-control-flow-15blocks.mir | 24 +-
.../NextUseAnalysis/complex-single-loop-a.mir | 24 +-
.../NextUseAnalysis/complex-single-loop.mir | 24 +-
.../double-nested-loops-complex-cfg.mir | 84 +-
...lse_with_loops_nested_in_2_outer_loops.mir | 42 +-
.../inner_cfg_in_2_nested_loops.mir | 12 +-
...op_nested_in_3_outer_loops_complex_cfg.mir | 36 +-
.../nested-loops-with-side-exits-a.mir | 48 +-
.../NextUseAnalysis/sequence_2_loops.mir | 12 +-
.../NextUseAnalysis/simple-loop-3blocks.mir | 6 +-
.../NextUseAnalysis/test_ers_basic_case.mir | 6 +-
...t_ers_do_not_spill_restore_inside_loop.mir | 18 +-
...t_ers_emit_restore_in_common_dominator.mir | 24 +-
...st_ers_emit_restore_in_loop_preheader1.mir | 6 +-
...st_ers_emit_restore_in_loop_preheader3.mir | 48 +-
...st_ers_emit_restore_in_loop_preheader4.mir | 12 +-
.../test_ers_keep_spilled_reg_live.mir | 6 +-
.../test_ers_multiple_spills1.mir | 24 +-
.../test_ers_multiple_spills2.mir | 18 +-
.../test_ers_multiple_spills3.mir | 30 +-
...common_dominator_and_optimize_restores.mir | 6 +-
.../test_ers_spill_loop_livethrough_reg.mir | 24 +-
...est_ers_spill_loop_value_in_exit_block.mir | 6 +-
.../NextUseAnalysis/triple-nested-loops.mir | 18 +-
.../NextUseAnalysis/two-sequential-loops.mir | 18 +-
.../AMDGPU/SRSRC-GIT-clobber-check.mir | 2 +-
.../AMDGPU/bb-prolog-spill-during-regalloc.ll | 16 +-
.../block-should-not-be-in-alive-blocks.mir | 6 +-
.../branch-folding-implicit-def-subreg.ll | 40 +-
.../coalescer-extend-pruned-subrange.mir | 4 +-
...scer-subranges-another-copymi-not-live.mir | 2 +-
...oalescer-subranges-another-prune-error.mir | 12 +-
.../AMDGPU/coalescer-subregjoin-fullcopy.mir | 4 +-
.../coalescer-with-subregs-bad-identical.mir | 4 +-
.../coalescing-with-subregs-in-loop-bug.mir | 2 +-
.../codegen-prepare-addrspacecast-non-null.ll | 14 +-
.../CodeGen/AMDGPU/collapse-endcf-broken.mir | 8 +-
llvm/test/CodeGen/AMDGPU/collapse-endcf.ll | 60 +-
llvm/test/CodeGen/AMDGPU/collapse-endcf.mir | 100 +--
llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir | 8 +-
.../AMDGPU/constant-fold-imm-immreg.mir | 2 +-
llvm/test/CodeGen/AMDGPU/convergent.mir | 36 +-
.../AMDGPU/cross-block-use-is-not-abi-copy.ll | 2 +-
llvm/test/CodeGen/AMDGPU/div_i128.ll | 158 ++--
llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir | 6 +-
llvm/test/CodeGen/AMDGPU/dpp_combine.mir | 2 +-
.../test/CodeGen/AMDGPU/dpp_combine_gfx11.mir | 2 +-
.../AMDGPU/extend-wwm-virt-reg-liveness.mir | 4 +-
.../CodeGen/AMDGPU/extract_vector_dynelt.ll | 117 ++-
.../fold-operands-s-add-copy-to-vgpr.mir | 4 +-
.../CodeGen/AMDGPU/fold-restore-undef-use.mir | 4 +-
.../greedy-alloc-fail-sgpr1024-spill.mir | 8 +-
.../greedy-broken-ssa-verifier-error.mir | 2 +-
.../CodeGen/AMDGPU/indirect-addressing-si.ll | 277 +++----
.../CodeGen/AMDGPU/insert-skip-from-vcc.mir | 10 +-
.../CodeGen/AMDGPU/insert-waitcnts-merge.ll | 4 +-
.../CodeGen/AMDGPU/insert_vector_dynelt.ll | 134 ++-
llvm/test/CodeGen/AMDGPU/licm-wwm.mir | 4 +-
...rval-bug-in-rename-independent-subregs.mir | 12 +-
.../AMDGPU/loop-header-align-gfx950.mir | 4 +-
.../lower-control-flow-live-intervals.mir | 22 +-
...wer-control-flow-live-variables-update.mir | 4 +-
.../lower-control-flow-other-terminators.mir | 2 +-
.../AMDGPU/lower-i1-copies-clear-kills.mir | 6 +-
...ne-scheduler-rematerialization-scoring.mir | 12 +-
.../machine-scheduler-sink-trivial-remats.mir | 48 +-
.../machine-sink-ignorable-exec-use.mir | 16 +-
.../CodeGen/AMDGPU/machine-sink-lane-mask.mir | 6 +-
...-var-out-of-divergent-loop-swdev407790.mir | 4 +-
...e-sink-temporal-divergence-swdev407790.mir | 16 +-
.../memory-legalizer-atomic-insert-end.mir | 4 +-
llvm/test/CodeGen/AMDGPU/merge-m0.mir | 6 +-
...uf-legalize-operands-non-ptr-intrinsics.ll | 125 ++-
.../CodeGen/AMDGPU/mubuf-legalize-operands.ll | 134 ++-
.../CodeGen/AMDGPU/no-remat-indirect-mov.mir | 4 +-
.../CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir | 6 +-
...ptimize-exec-mask-pre-ra-alloc-failure.mir | 4 +-
...ptimize-exec-mask-pre-ra-def-after-use.mir | 10 +-
...ask-pre-ra-non-empty-but-used-interval.mir | 4 +-
.../AMDGPU/optimize-exec-masking-pre-ra.mir | 16 +-
.../AMDGPU/optimize-if-exec-masking.mir | 22 +-
.../AMDGPU/partial-sgpr-to-vgpr-spills.ll | 33 +-
.../CodeGen/AMDGPU/phi-elimination-end-cf.mir | 6 +-
.../AMDGPU/postra-sink-update-dependency.mir | 6 +-
.../ran-out-of-sgprs-allocation-failure.mir | 72 +-
llvm/test/CodeGen/AMDGPU/readlane_exec0.mir | 2 +-
...oc-hoist-spill-live-range-interference.mir | 13 +-
.../regalloc-hoist-spill-live-range-upd.mir | 4 +-
llvm/test/CodeGen/AMDGPU/rem_i128.ll | 78 +-
...hort-exec-branch-on-unconditional-jump.mir | 12 +-
.../AMDGPU/s_or_saveexec_xor_combine.mir | 24 +-
.../CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll | 7 +-
.../AMDGPU/sgpr-spill-overlap-wwm-reserve.mir | 150 ++--
.../CodeGen/AMDGPU/si-fix-sgpr-copies.mir | 2 +-
.../CodeGen/AMDGPU/si-init-whole-wave.mir | 8 +-
...i-lower-control-flow-preserve-dom-tree.mir | 4 +-
...w-remove-redundant-block-liveintervals.mir | 6 +-
...lower-i1-copies-order-of-phi-incomings.mir | 2 +-
.../CodeGen/AMDGPU/si-lower-i1-copies.mir | 4 +-
.../si-opt-vgpr-liverange-bug-deadlanes.mir | 2 +-
...-opt-vgpr-liverange-undef-use-livevars.mir | 2 +-
.../si-optimize-vgpr-live-range-dbg-instr.mir | 2 +-
.../si-pre-allocate-wwm-regs-preserve-rci.mir | 4 +-
.../AMDGPU/sink-after-control-flow-postra.mir | 48 +-
.../AMDGPU/sink-after-control-flow.mir | 12 +-
.../AMDGPU/skip-branch-taildup-ret.mir | 24 +-
.../test/CodeGen/AMDGPU/spill-before-exec.mir | 6 +-
.../CodeGen/AMDGPU/spill-scavenge-offset.ll | 459 ++++++-----
.../AMDGPU/stale-livevar-in-twoaddr-pass.mir | 2 +-
.../stop-tail-duplicate-cfg-intrinsic.mir | 2 +-
.../AMDGPU/subreg-split-live-in-error.mir | 2 +-
.../transform-block-with-return-to-epilog.ll | 6 +-
llvm/test/CodeGen/AMDGPU/trap-abis.ll | 5 +-
.../AMDGPU/undefined-physreg-sgpr-spill.mir | 8 +-
.../CodeGen/AMDGPU/v-cmp-cse-across-loop.mir | 2 +-
.../AMDGPU/vgpr-mark-last-scratch-load.mir | 8 +-
.../waitcnt-kmcnt-scc-different-block.mir | 2 +-
...terfall-call-target-av-register-failure.ll | 11 +-
.../AMDGPU/whole-wave-functions-pei.mir | 6 +-
.../AMDGPU/wqm-debug-instr-terminator.mir | 6 +-
llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir | 2 +-
.../wqm-propagate-for-execz-side-effect.mir | 24 +-
llvm/test/CodeGen/AMDGPU/wqm-terminators.mir | 2 +-
llvm/test/CodeGen/AMDGPU/wqm.mir | 77 +-
.../wwm-regalloc-preallocation-guard.mir | 4 +-
.../test/CodeGen/AMDGPU/wwm-reserved-spill.ll | 4 +-
llvm/test/CodeGen/AMDGPU/wwm-reserved.ll | 8 +-
.../AMDGPU/wwm-spill-superclass-pseudo.mir | 2 +-
136 files changed, 2107 insertions(+), 2111 deletions(-)
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll
index 33d7bd4c75ace0..cd304a7fd931fb 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll
@@ -66,9 +66,6 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: buffer_store_dword v16, off, s[0:3], s32 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b32 exec_lo, s21
; CHECK-NEXT: .LBB0_1: ; =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: s_or_saveexec_b32 s21, -1
-; CHECK-NEXT: buffer_load_dword v16, off, s[0:3], s32 ; 4-byte Folded Reload
-; CHECK-NEXT: s_mov_b32 exec_lo, s21
; CHECK-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
@@ -85,7 +82,10 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; CHECK-NEXT: s_waitcnt vmcnt(0)
+; CHECK-NEXT: s_or_saveexec_b32 s21, -1
+; CHECK-NEXT: buffer_load_dword v16, off, s[0:3], s32 ; 4-byte Folded Reload
+; CHECK-NEXT: s_mov_b32 exec_lo, s21
+; CHECK-NEXT: s_waitcnt vmcnt(1)
; CHECK-NEXT: v_readfirstlane_b32 s12, v7
; CHECK-NEXT: v_readfirstlane_b32 s10, v6
; CHECK-NEXT: v_readfirstlane_b32 s9, v5
@@ -102,6 +102,7 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: s_mov_b32 s17, s6
; CHECK-NEXT: s_mov_b32 s18, s5
; CHECK-NEXT: s_mov_b32 s19, s4
+; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: v_writelane_b32 v16, s12, 5
; CHECK-NEXT: v_writelane_b32 v16, s13, 6
; CHECK-NEXT: v_writelane_b32 v16, s14, 7
@@ -135,6 +136,8 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: buffer_store_dword v16, off, s[0:3], s32 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b32 exec_lo, s21
; CHECK-NEXT: ; %bb.2: ; in Loop: Header=BB0_1 Depth=1
+; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; CHECK-NEXT: s_or_saveexec_b32 s21, -1
; CHECK-NEXT: buffer_load_dword v16, off, s[0:3], s32 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b32 exec_lo, s21
@@ -152,9 +155,6 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: v_readlane_b32 s17, v16, 1
; CHECK-NEXT: v_readlane_b32 s18, v16, 2
; CHECK-NEXT: v_readlane_b32 s19, v16, 3
-; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: image_sample v0, [v0, v1], s[8:15], s[16:19] dmask:0x1 dim:SQ_RSRC_IMG_2D
; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll
index 02b68822047c3b..7b817e1c91eaa0 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll
@@ -667,12 +667,10 @@ define amdgpu_ps void @wqm_deriv(<2 x float> %input, float %arg, i32 %index) {
; SI-NEXT: s_wqm_b64 exec, exec
; SI-NEXT: v_cvt_i32_f32_e32 v0, v0
; SI-NEXT: s_movk_i32 s2, 0x3c00
-; SI-NEXT: s_bfe_u32 s3, 0, 0x100000
-; SI-NEXT: s_bfe_u32 s2, s2, 0x100000
-; SI-NEXT: s_lshl_b32 s4, s3, 16
-; SI-NEXT: s_or_b32 s4, s2, s4
-; SI-NEXT: s_lshl_b32 s2, s2, 16
-; SI-NEXT: s_or_b32 s5, s3, s2
+; SI-NEXT: s_bfe_u32 s4, s2, 0x100000
+; SI-NEXT: s_bfe_u32 s5, 0, 0x100000
+; SI-NEXT: s_lshl_b32 s6, s5, 16
+; SI-NEXT: s_lshl_b32 s7, s4, 16
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; SI-NEXT: s_and_saveexec_b64 s[2:3], vcc
; SI-NEXT: s_xor_b64 s[2:3], exec, s[2:3]
@@ -681,10 +679,12 @@ define amdgpu_ps void @wqm_deriv(<2 x float> %input, float %arg, i32 %index) {
; SI-NEXT: s_andn2_b64 s[0:1], s[0:1], exec
; SI-NEXT: s_cbranch_scc0 .LBB6_7
; SI-NEXT: ; %bb.2: ; %.demote0
-; SI-NEXT: s_wqm_b64 s[6:7], s[0:1]
-; SI-NEXT: s_and_b64 exec, exec, s[6:7]
+; SI-NEXT: s_wqm_b64 s[8:9], s[0:1]
+; SI-NEXT: s_and_b64 exec, exec, s[8:9]
; SI-NEXT: .LBB6_3: ; %.continue0
; SI-NEXT: s_or_b64 exec, exec, s[2:3]
+; SI-NEXT: s_or_b32 s4, s4, s6
+; SI-NEXT: s_or_b32 s5, s5, s7
; SI-NEXT: s_mov_b64 s[6:7], s[0:1]
; SI-NEXT: v_cndmask_b32_e64 v0, 1.0, 0, s[6:7]
; SI-NEXT: v_mov_b32_e32 v1, v0
@@ -901,13 +901,11 @@ define amdgpu_ps void @wqm_deriv_loop(<2 x float> %input, float %arg, i32 %index
; SI-NEXT: s_wqm_b64 exec, exec
; SI-NEXT: v_cvt_i32_f32_e32 v0, v0
; SI-NEXT: s_movk_i32 s2, 0x3c00
-; SI-NEXT: s_bfe_u32 s3, 0, 0x100000
-; SI-NEXT: s_bfe_u32 s2, s2, 0x100000
-; SI-NEXT: s_lshl_b32 s4, s3, 16
-; SI-NEXT: s_or_b32 s6, s2, s4
-; SI-NEXT: s_lshl_b32 s2, s2, 16
-; SI-NEXT: s_or_b32 s7, s3, s2
-; SI-NEXT: s_mov_b32 s8, 0
+; SI-NEXT: s_bfe_u32 s4, s2, 0x100000
+; SI-NEXT: s_bfe_u32 s5, 0, 0x100000
+; SI-NEXT: s_lshl_b32 s7, s5, 16
+; SI-NEXT: s_lshl_b32 s8, s4, 16
+; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; SI-NEXT: s_and_saveexec_b64 s[2:3], vcc
; SI-NEXT: s_xor_b64 s[2:3], exec, s[2:3]
@@ -916,23 +914,25 @@ define amdgpu_ps void @wqm_deriv_loop(<2 x float> %input, float %arg, i32 %index
; SI-NEXT: s_andn2_b64 s[0:1], s[0:1], exec
; SI-NEXT: s_cbranch_scc0 .LBB7_9
; SI-NEXT: ; %bb.2: ; %.demote0
-; SI-NEXT: s_wqm_b64 s[4:5], s[0:1]
-; SI-NEXT: s_and_b64 exec, exec, s[4:5]
+; SI-NEXT: s_wqm_b64 s[10:11], s[0:1]
+; SI-NEXT: s_and_b64 exec, exec, s[10:11]
; SI-NEXT: .LBB7_3: ; %.continue0.preheader
; SI-NEXT: s_or_b64 exec, exec, s[2:3]
+; SI-NEXT: s_or_b32 s7, s4, s7
+; SI-NEXT: s_or_b32 s8, s5, s8
; SI-NEXT: s_mov_b64 s[2:3], 0
; SI-NEXT: s_branch .LBB7_5
; SI-NEXT: .LBB7_4: ; %.continue1
; SI-NEXT: ; in Loop: Header=BB7_5 Depth=1
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: s_add_i32 s8, s8, 1
-; SI-NEXT: v_cmp_ge_i32_e32 vcc, s8, v1
+; SI-NEXT: s_add_i32 s6, s6, 1
+; SI-NEXT: v_cmp_ge_i32_e32 vcc, s6, v1
; SI-NEXT: s_or_b64 s[2:3], vcc, s[2:3]
; SI-NEXT: s_andn2_b64 exec, exec, s[2:3]
; SI-NEXT: s_cbranch_execz .LBB7_8
; SI-NEXT: .LBB7_5: ; %.continue0
; SI-NEXT: ; =>This Inner Loop Header: Depth=1
-; SI-NEXT: v_mov_b32_e32 v0, s8
+; SI-NEXT: v_mov_b32_e32 v0, s6
; SI-NEXT: s_mov_b64 s[10:11], s[0:1]
; SI-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[10:11]
; SI-NEXT: v_mov_b32_e32 v2, v0
@@ -960,8 +960,8 @@ define amdgpu_ps void @wqm_deriv_loop(<2 x float> %input, float %arg, i32 %index
; SI-NEXT: .LBB7_8: ; %.return
; SI-NEXT: s_or_b64 exec, exec, s[2:3]
; SI-NEXT: s_and_b64 exec, exec, s[0:1]
-; SI-NEXT: v_mov_b32_e32 v0, s6
-; SI-NEXT: v_mov_b32_e32 v1, s7
+; SI-NEXT: v_mov_b32_e32 v0, s7
+; SI-NEXT: v_mov_b32_e32 v1, s8
; SI-NEXT: exp mrt0, v0, v0, v1, v1 done compr vm
; SI-NEXT: s_endpgm
; SI-NEXT: .LBB7_9:
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir
index d0fbac674e5238..b576c3f1857759 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir
@@ -89,7 +89,7 @@ body: |
successors: %bb.7(0x80000000)
%28:vgpr_32 = PHI %25, %bb.4, %27, %bb.5
- SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.7:
successors: %bb.8(0x80000000)
@@ -134,7 +134,7 @@ body: |
successors: %bb.13(0x80000000)
%44:vgpr_32 = PHI %41, %bb.10, %43, %bb.11
- SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.13:
@@ -933,7 +933,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1455,7 +1455,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2449,7 +2449,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2967,7 +2967,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir
index 1085000dd83ec7..d9befc88206e07 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir
@@ -89,7 +89,7 @@ body: |
successors: %bb.7(0x80000000)
%28:vgpr_32 = PHI %25, %bb.4, %27, %bb.5
- SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.7:
successors: %bb.8(0x80000000)
@@ -134,7 +134,7 @@ body: |
successors: %bb.13(0x80000000)
%44:vgpr_32 = PHI %41, %bb.10, %43, %bb.11
- SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.13:
successors: %bb.14(0x80000000)
@@ -179,7 +179,7 @@ body: |
successors: %bb.19(0x80000000)
%60:vgpr_32 = PHI %57, %bb.16, %59, %bb.17
- SI_END_CF killed %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.19:
successors: %bb.20(0x80000000)
@@ -224,7 +224,7 @@ body: |
successors: %bb.25(0x80000000)
%76:vgpr_32 = PHI %73, %bb.22, %75, %bb.23
- SI_END_CF killed %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.25:
successors: %bb.26(0x80000000)
@@ -269,7 +269,7 @@ body: |
successors: %bb.31(0x80000000)
%92:vgpr_32 = PHI %89, %bb.28, %91, %bb.29
- SI_END_CF killed %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.31:
successors: %bb.32(0x80000000)
@@ -314,7 +314,7 @@ body: |
successors: %bb.37(0x80000000)
%108:vgpr_32 = PHI %105, %bb.34, %107, %bb.35
- SI_END_CF killed %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.37:
successors: %bb.38(0x80000000)
@@ -359,7 +359,7 @@ body: |
successors: %bb.43(0x80000000)
%124:vgpr_32 = PHI %121, %bb.40, %123, %bb.41
- SI_END_CF killed %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.43:
successors: %bb.44(0x80000000)
@@ -404,7 +404,7 @@ body: |
successors: %bb.49(0x80000000)
%140:vgpr_32 = PHI %137, %bb.46, %139, %bb.47
- SI_END_CF killed %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.49:
successors: %bb.50(0x80000000)
@@ -449,7 +449,7 @@ body: |
successors: %bb.55(0x80000000)
%156:vgpr_32 = PHI %153, %bb.52, %155, %bb.53
- SI_END_CF killed %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.55:
successors: %bb.56(0x80000000)
@@ -494,7 +494,7 @@ body: |
successors: %bb.61(0x80000000)
%172:vgpr_32 = PHI %169, %bb.58, %171, %bb.59
- SI_END_CF killed %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.61:
successors: %bb.62(0x80000000)
@@ -539,7 +539,7 @@ body: |
successors: %bb.67(0x80000000)
%188:vgpr_32 = PHI %185, %bb.64, %187, %bb.65
- SI_END_CF killed %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.67:
successors: %bb.68(0x80000000)
@@ -584,7 +584,7 @@ body: |
successors: %bb.73(0x80000000)
%204:vgpr_32 = PHI %201, %bb.70, %203, %bb.71
- SI_END_CF killed %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.73:
successors: %bb.74(0x80000000)
@@ -629,7 +629,7 @@ body: |
successors: %bb.79(0x80000000)
%220:vgpr_32 = PHI %217, %bb.76, %219, %bb.77
- SI_END_CF killed %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.79:
successors: %bb.80(0x80000000)
@@ -674,7 +674,7 @@ body: |
successors: %bb.85(0x80000000)
%236:vgpr_32 = PHI %233, %bb.82, %235, %bb.83
- SI_END_CF killed %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.85:
successors: %bb.86(0x80000000)
@@ -719,7 +719,7 @@ body: |
successors: %bb.91(0x80000000)
%252:vgpr_32 = PHI %249, %bb.88, %251, %bb.89
- SI_END_CF killed %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.91:
successors: %bb.92(0x80000000)
@@ -764,7 +764,7 @@ body: |
successors: %bb.97(0x80000000)
%268:vgpr_32 = PHI %265, %bb.94, %267, %bb.95
- SI_END_CF killed %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.97:
successors: %bb.98(0x80000000)
@@ -809,7 +809,7 @@ body: |
successors: %bb.103(0x80000000)
%284:vgpr_32 = PHI %281, %bb.100, %283, %bb.101
- SI_END_CF killed %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.103:
successors: %bb.104(0x80000000)
@@ -854,7 +854,7 @@ body: |
successors: %bb.109(0x80000000)
%300:vgpr_32 = PHI %297, %bb.106, %299, %bb.107
- SI_END_CF killed %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.109:
successors: %bb.110(0x80000000)
@@ -899,7 +899,7 @@ body: |
successors: %bb.115(0x80000000)
%316:vgpr_32 = PHI %313, %bb.112, %315, %bb.113
- SI_END_CF killed %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.115:
successors: %bb.116(0x80000000)
@@ -944,7 +944,7 @@ body: |
successors: %bb.121(0x80000000)
%332:vgpr_32 = PHI %329, %bb.118, %331, %bb.119
- SI_END_CF killed %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.121:
successors: %bb.122(0x80000000)
@@ -989,7 +989,7 @@ body: |
successors: %bb.127(0x80000000)
%348:vgpr_32 = PHI %345, %bb.124, %347, %bb.125
- SI_END_CF killed %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.127:
successors: %bb.128(0x80000000)
@@ -1034,7 +1034,7 @@ body: |
successors: %bb.133(0x80000000)
%364:vgpr_32 = PHI %361, %bb.130, %363, %bb.131
- SI_END_CF killed %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.133:
successors: %bb.134(0x80000000)
@@ -1079,7 +1079,7 @@ body: |
successors: %bb.139(0x80000000)
%380:vgpr_32 = PHI %377, %bb.136, %379, %bb.137
- SI_END_CF killed %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.139:
successors: %bb.140(0x80000000)
@@ -1124,7 +1124,7 @@ body: |
successors: %bb.145(0x80000000)
%396:vgpr_32 = PHI %393, %bb.142, %395, %bb.143
- SI_END_CF killed %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.145:
successors: %bb.146(0x80000000)
@@ -1169,7 +1169,7 @@ body: |
successors: %bb.151(0x80000000)
%412:vgpr_32 = PHI %409, %bb.148, %411, %bb.149
- SI_END_CF killed %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.151:
successors: %bb.152(0x80000000)
@@ -1214,7 +1214,7 @@ body: |
successors: %bb.157(0x80000000)
%428:vgpr_32 = PHI %425, %bb.154, %427, %bb.155
- SI_END_CF killed %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.157:
successors: %bb.158(0x80000000)
@@ -1259,7 +1259,7 @@ body: |
successors: %bb.163(0x80000000)
%444:vgpr_32 = PHI %441, %bb.160, %443, %bb.161
- SI_END_CF killed %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.163:
successors: %bb.164(0x80000000)
@@ -1304,7 +1304,7 @@ body: |
successors: %bb.169(0x80000000)
%460:vgpr_32 = PHI %457, %bb.166, %459, %bb.167
- SI_END_CF killed %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.169:
successors: %bb.170(0x80000000)
@@ -1349,7 +1349,7 @@ body: |
successors: %bb.175(0x80000000)
%476:vgpr_32 = PHI %473, %bb.172, %475, %bb.173
- SI_END_CF killed %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.175:
successors: %bb.176(0x80000000)
@@ -1394,7 +1394,7 @@ body: |
successors: %bb.181(0x80000000)
%492:vgpr_32 = PHI %489, %bb.178, %491, %bb.179
- SI_END_CF killed %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.181:
successors: %bb.182(0x80000000)
@@ -1439,7 +1439,7 @@ body: |
successors: %bb.187(0x80000000)
%508:vgpr_32 = PHI %505, %bb.184, %507, %bb.185
- SI_END_CF killed %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.187:
successors: %bb.188(0x80000000)
@@ -1484,7 +1484,7 @@ body: |
successors: %bb.193(0x80000000)
%524:vgpr_32 = PHI %521, %bb.190, %523, %bb.191
- SI_END_CF killed %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.193:
successors: %bb.194(0x80000000)
@@ -1529,7 +1529,7 @@ body: |
successors: %bb.199(0x80000000)
%540:vgpr_32 = PHI %537, %bb.196, %539, %bb.197
- SI_END_CF killed %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.199:
successors: %bb.200(0x80000000)
@@ -1574,7 +1574,7 @@ body: |
successors: %bb.205(0x80000000)
%556:vgpr_32 = PHI %553, %bb.202, %555, %bb.203
- SI_END_CF killed %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.205:
successors: %bb.206(0x80000000)
@@ -1619,7 +1619,7 @@ body: |
successors: %bb.211(0x80000000)
%572:vgpr_32 = PHI %569, %bb.208, %571, %bb.209
- SI_END_CF killed %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.211:
successors: %bb.212(0x80000000)
@@ -1664,7 +1664,7 @@ body: |
successors: %bb.217(0x80000000)
%588:vgpr_32 = PHI %585, %bb.214, %587, %bb.215
- SI_END_CF killed %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.217:
successors: %bb.218(0x80000000)
@@ -1709,7 +1709,7 @@ body: |
successors: %bb.223(0x80000000)
%604:vgpr_32 = PHI %601, %bb.220, %603, %bb.221
- SI_END_CF killed %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.223:
successors: %bb.224(0x80000000)
@@ -1754,7 +1754,7 @@ body: |
successors: %bb.229(0x80000000)
%620:vgpr_32 = PHI %617, %bb.226, %619, %bb.227
- SI_END_CF killed %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.229:
successors: %bb.230(0x80000000)
@@ -1799,7 +1799,7 @@ body: |
successors: %bb.235(0x80000000)
%636:vgpr_32 = PHI %633, %bb.232, %635, %bb.233
- SI_END_CF killed %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.235:
successors: %bb.236(0x80000000)
@@ -1844,7 +1844,7 @@ body: |
successors: %bb.241(0x80000000)
%652:vgpr_32 = PHI %649, %bb.238, %651, %bb.239
- SI_END_CF killed %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.241:
successors: %bb.242(0x80000000)
@@ -1889,7 +1889,7 @@ body: |
successors: %bb.247(0x80000000)
%668:vgpr_32 = PHI %665, %bb.244, %667, %bb.245
- SI_END_CF killed %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.247:
successors: %bb.248(0x80000000)
@@ -1934,7 +1934,7 @@ body: |
successors: %bb.253(0x80000000)
%684:vgpr_32 = PHI %681, %bb.250, %683, %bb.251
- SI_END_CF killed %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.253:
successors: %bb.254(0x80000000)
@@ -1979,7 +1979,7 @@ body: |
successors: %bb.259(0x80000000)
%700:vgpr_32 = PHI %697, %bb.256, %699, %bb.257
- SI_END_CF killed %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.259:
successors: %bb.260(0x80000000)
@@ -2024,7 +2024,7 @@ body: |
successors: %bb.265(0x80000000)
%716:vgpr_32 = PHI %713, %bb.262, %715, %bb.263
- SI_END_CF killed %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.265:
successors: %bb.266(0x80000000)
@@ -2069,7 +2069,7 @@ body: |
successors: %bb.271(0x80000000)
%732:vgpr_32 = PHI %729, %bb.268, %731, %bb.269
- SI_END_CF killed %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.271:
successors: %bb.272(0x80000000)
@@ -2114,7 +2114,7 @@ body: |
successors: %bb.277(0x80000000)
%748:vgpr_32 = PHI %745, %bb.274, %747, %bb.275
- SI_END_CF killed %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.277:
successors: %bb.278(0x80000000)
@@ -2159,7 +2159,7 @@ body: |
successors: %bb.283(0x80000000)
%764:vgpr_32 = PHI %761, %bb.280, %763, %bb.281
- SI_END_CF killed %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.283:
successors: %bb.284(0x80000000)
@@ -2204,7 +2204,7 @@ body: |
successors: %bb.289(0x80000000)
%780:vgpr_32 = PHI %777, %bb.286, %779, %bb.287
- SI_END_CF killed %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.289:
successors: %bb.290(0x80000000)
@@ -2249,7 +2249,7 @@ body: |
successors: %bb.295(0x80000000)
%796:vgpr_32 = PHI %793, %bb.292, %795, %bb.293
- SI_END_CF killed %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.295:
successors: %bb.296(0x80000000)
@@ -2294,7 +2294,7 @@ body: |
successors: %bb.301(0x80000000)
%812:vgpr_32 = PHI %809, %bb.298, %811, %bb.299
- SI_END_CF killed %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.301:
successors: %bb.302(0x80000000)
@@ -2339,7 +2339,7 @@ body: |
successors: %bb.307(0x80000000)
%828:vgpr_32 = PHI %825, %bb.304, %827, %bb.305
- SI_END_CF killed %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.307:
successors: %bb.308(0x80000000)
@@ -2384,7 +2384,7 @@ body: |
successors: %bb.313(0x80000000)
%844:vgpr_32 = PHI %841, %bb.310, %843, %bb.311
- SI_END_CF killed %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.313:
successors: %bb.314(0x80000000)
@@ -2429,7 +2429,7 @@ body: |
successors: %bb.319(0x80000000)
%860:vgpr_32 = PHI %857, %bb.316, %859, %bb.317
- SI_END_CF killed %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.319:
successors: %bb.320(0x80000000)
@@ -2474,7 +2474,7 @@ body: |
successors: %bb.325(0x80000000)
%876:vgpr_32 = PHI %873, %bb.322, %875, %bb.323
- SI_END_CF killed %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.325:
successors: %bb.326(0x80000000)
@@ -2519,7 +2519,7 @@ body: |
successors: %bb.331(0x80000000)
%892:vgpr_32 = PHI %889, %bb.328, %891, %bb.329
- SI_END_CF killed %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.331:
successors: %bb.332(0x80000000)
@@ -2564,7 +2564,7 @@ body: |
successors: %bb.337(0x80000000)
%908:vgpr_32 = PHI %905, %bb.334, %907, %bb.335
- SI_END_CF killed %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.337:
successors: %bb.338(0x80000000)
@@ -2609,7 +2609,7 @@ body: |
successors: %bb.343(0x80000000)
%924:vgpr_32 = PHI %921, %bb.340, %923, %bb.341
- SI_END_CF killed %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.343:
successors: %bb.344(0x80000000)
@@ -2654,7 +2654,7 @@ body: |
successors: %bb.349(0x80000000)
%940:vgpr_32 = PHI %937, %bb.346, %939, %bb.347
- SI_END_CF killed %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.349:
successors: %bb.350(0x80000000)
@@ -2699,7 +2699,7 @@ body: |
successors: %bb.355(0x80000000)
%956:vgpr_32 = PHI %953, %bb.352, %955, %bb.353
- SI_END_CF killed %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.355:
successors: %bb.356(0x80000000)
@@ -2744,7 +2744,7 @@ body: |
successors: %bb.361(0x80000000)
%972:vgpr_32 = PHI %969, %bb.358, %971, %bb.359
- SI_END_CF killed %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.361:
successors: %bb.362(0x80000000)
@@ -2789,7 +2789,7 @@ body: |
successors: %bb.367(0x80000000)
%988:vgpr_32 = PHI %985, %bb.364, %987, %bb.365
- SI_END_CF killed %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.367:
successors: %bb.368(0x80000000)
@@ -2834,7 +2834,7 @@ body: |
successors: %bb.373(0x80000000)
%1004:vgpr_32 = PHI %1001, %bb.370, %1003, %bb.371
- SI_END_CF killed %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.373:
successors: %bb.374(0x80000000)
@@ -2879,7 +2879,7 @@ body: |
successors: %bb.379(0x80000000)
%1020:vgpr_32 = PHI %1017, %bb.376, %1019, %bb.377
- SI_END_CF killed %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.379:
successors: %bb.380(0x80000000)
@@ -2924,7 +2924,7 @@ body: |
successors: %bb.385(0x80000000)
%1036:vgpr_32 = PHI %1033, %bb.382, %1035, %bb.383
- SI_END_CF killed %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.385:
successors: %bb.386(0x80000000)
@@ -2969,7 +2969,7 @@ body: |
successors: %bb.391(0x80000000)
%1052:vgpr_32 = PHI %1049, %bb.388, %1051, %bb.389
- SI_END_CF killed %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.391:
successors: %bb.392(0x80000000)
@@ -3014,7 +3014,7 @@ body: |
successors: %bb.397(0x80000000)
%1068:vgpr_32 = PHI %1065, %bb.394, %1067, %bb.395
- SI_END_CF killed %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.397:
successors: %bb.398(0x80000000)
@@ -3059,7 +3059,7 @@ body: |
successors: %bb.403(0x80000000)
%1084:vgpr_32 = PHI %1081, %bb.400, %1083, %bb.401
- SI_END_CF killed %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.403:
successors: %bb.404(0x80000000)
@@ -3104,7 +3104,7 @@ body: |
successors: %bb.409(0x80000000)
%1100:vgpr_32 = PHI %1097, %bb.406, %1099, %bb.407
- SI_END_CF killed %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.409:
successors: %bb.410(0x80000000)
@@ -3149,7 +3149,7 @@ body: |
successors: %bb.415(0x80000000)
%1116:vgpr_32 = PHI %1113, %bb.412, %1115, %bb.413
- SI_END_CF killed %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.415:
successors: %bb.416(0x80000000)
@@ -3194,7 +3194,7 @@ body: |
successors: %bb.421(0x80000000)
%1132:vgpr_32 = PHI %1129, %bb.418, %1131, %bb.419
- SI_END_CF killed %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.421:
successors: %bb.422(0x80000000)
@@ -3239,7 +3239,7 @@ body: |
successors: %bb.427(0x80000000)
%1148:vgpr_32 = PHI %1145, %bb.424, %1147, %bb.425
- SI_END_CF killed %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.427:
successors: %bb.428(0x80000000)
@@ -3284,7 +3284,7 @@ body: |
successors: %bb.433(0x80000000)
%1164:vgpr_32 = PHI %1161, %bb.430, %1163, %bb.431
- SI_END_CF killed %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.433:
successors: %bb.434(0x80000000)
@@ -3329,7 +3329,7 @@ body: |
successors: %bb.439(0x80000000)
%1180:vgpr_32 = PHI %1177, %bb.436, %1179, %bb.437
- SI_END_CF killed %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.439:
successors: %bb.440(0x80000000)
@@ -3374,7 +3374,7 @@ body: |
successors: %bb.445(0x80000000)
%1196:vgpr_32 = PHI %1193, %bb.442, %1195, %bb.443
- SI_END_CF killed %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.445:
successors: %bb.446(0x80000000)
@@ -3419,7 +3419,7 @@ body: |
successors: %bb.451(0x80000000)
%1212:vgpr_32 = PHI %1209, %bb.448, %1211, %bb.449
- SI_END_CF killed %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.451:
successors: %bb.452(0x80000000)
@@ -3464,7 +3464,7 @@ body: |
successors: %bb.457(0x80000000)
%1228:vgpr_32 = PHI %1225, %bb.454, %1227, %bb.455
- SI_END_CF killed %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.457:
successors: %bb.458(0x80000000)
@@ -3509,7 +3509,7 @@ body: |
successors: %bb.463(0x80000000)
%1244:vgpr_32 = PHI %1241, %bb.460, %1243, %bb.461
- SI_END_CF killed %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.463:
successors: %bb.464(0x80000000)
@@ -3554,7 +3554,7 @@ body: |
successors: %bb.469(0x80000000)
%1260:vgpr_32 = PHI %1257, %bb.466, %1259, %bb.467
- SI_END_CF killed %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.469:
successors: %bb.470(0x80000000)
@@ -3599,7 +3599,7 @@ body: |
successors: %bb.475(0x80000000)
%1276:vgpr_32 = PHI %1273, %bb.472, %1275, %bb.473
- SI_END_CF killed %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.475:
successors: %bb.476(0x80000000)
@@ -3644,7 +3644,7 @@ body: |
successors: %bb.481(0x80000000)
%1292:vgpr_32 = PHI %1289, %bb.478, %1291, %bb.479
- SI_END_CF killed %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.481:
successors: %bb.482(0x80000000)
@@ -3689,7 +3689,7 @@ body: |
successors: %bb.487(0x80000000)
%1308:vgpr_32 = PHI %1305, %bb.484, %1307, %bb.485
- SI_END_CF killed %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.487:
successors: %bb.488(0x80000000)
@@ -3734,7 +3734,7 @@ body: |
successors: %bb.493(0x80000000)
%1324:vgpr_32 = PHI %1321, %bb.490, %1323, %bb.491
- SI_END_CF killed %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.493:
successors: %bb.494(0x80000000)
@@ -3779,7 +3779,7 @@ body: |
successors: %bb.499(0x80000000)
%1340:vgpr_32 = PHI %1337, %bb.496, %1339, %bb.497
- SI_END_CF killed %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.499:
successors: %bb.500(0x80000000)
@@ -3824,7 +3824,7 @@ body: |
successors: %bb.505(0x80000000)
%1356:vgpr_32 = PHI %1353, %bb.502, %1355, %bb.503
- SI_END_CF killed %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.505:
successors: %bb.506(0x80000000)
@@ -3869,7 +3869,7 @@ body: |
successors: %bb.511(0x80000000)
%1372:vgpr_32 = PHI %1369, %bb.508, %1371, %bb.509
- SI_END_CF killed %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.511:
successors: %bb.512(0x80000000)
@@ -3914,7 +3914,7 @@ body: |
successors: %bb.517(0x80000000)
%1388:vgpr_32 = PHI %1385, %bb.514, %1387, %bb.515
- SI_END_CF killed %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.517:
successors: %bb.518(0x80000000)
@@ -3959,7 +3959,7 @@ body: |
successors: %bb.523(0x80000000)
%1404:vgpr_32 = PHI %1401, %bb.520, %1403, %bb.521
- SI_END_CF killed %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.523:
successors: %bb.524(0x80000000)
@@ -4004,7 +4004,7 @@ body: |
successors: %bb.529(0x80000000)
%1420:vgpr_32 = PHI %1417, %bb.526, %1419, %bb.527
- SI_END_CF killed %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.529:
successors: %bb.530(0x80000000)
@@ -4049,7 +4049,7 @@ body: |
successors: %bb.535(0x80000000)
%1436:vgpr_32 = PHI %1433, %bb.532, %1435, %bb.533
- SI_END_CF killed %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.535:
successors: %bb.536(0x80000000)
@@ -4094,7 +4094,7 @@ body: |
successors: %bb.541(0x80000000)
%1452:vgpr_32 = PHI %1449, %bb.538, %1451, %bb.539
- SI_END_CF killed %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.541:
successors: %bb.542(0x80000000)
@@ -4139,7 +4139,7 @@ body: |
successors: %bb.547(0x80000000)
%1468:vgpr_32 = PHI %1465, %bb.544, %1467, %bb.545
- SI_END_CF killed %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.547:
successors: %bb.548(0x80000000)
@@ -4184,7 +4184,7 @@ body: |
successors: %bb.553(0x80000000)
%1484:vgpr_32 = PHI %1481, %bb.550, %1483, %bb.551
- SI_END_CF killed %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.553:
successors: %bb.554(0x80000000)
@@ -4229,7 +4229,7 @@ body: |
successors: %bb.559(0x80000000)
%1500:vgpr_32 = PHI %1497, %bb.556, %1499, %bb.557
- SI_END_CF killed %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.559:
successors: %bb.560(0x80000000)
@@ -4274,7 +4274,7 @@ body: |
successors: %bb.565(0x80000000)
%1516:vgpr_32 = PHI %1513, %bb.562, %1515, %bb.563
- SI_END_CF killed %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.565:
successors: %bb.566(0x80000000)
@@ -4319,7 +4319,7 @@ body: |
successors: %bb.571(0x80000000)
%1532:vgpr_32 = PHI %1529, %bb.568, %1531, %bb.569
- SI_END_CF killed %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.571:
successors: %bb.572(0x80000000)
@@ -4364,7 +4364,7 @@ body: |
successors: %bb.577(0x80000000)
%1548:vgpr_32 = PHI %1545, %bb.574, %1547, %bb.575
- SI_END_CF killed %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.577:
successors: %bb.578(0x80000000)
@@ -4409,7 +4409,7 @@ body: |
successors: %bb.583(0x80000000)
%1564:vgpr_32 = PHI %1561, %bb.580, %1563, %bb.581
- SI_END_CF killed %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.583:
successors: %bb.584(0x80000000)
@@ -4454,7 +4454,7 @@ body: |
successors: %bb.589(0x80000000)
%1580:vgpr_32 = PHI %1577, %bb.586, %1579, %bb.587
- SI_END_CF killed %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.589:
successors: %bb.590(0x80000000)
@@ -4499,7 +4499,7 @@ body: |
successors: %bb.595(0x80000000)
%1596:vgpr_32 = PHI %1593, %bb.592, %1595, %bb.593
- SI_END_CF killed %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.595:
successors: %bb.596(0x80000000)
@@ -4544,7 +4544,7 @@ body: |
successors: %bb.601(0x80000000)
%1612:vgpr_32 = PHI %1609, %bb.598, %1611, %bb.599
- SI_END_CF killed %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.601:
successors: %bb.602(0x80000000)
@@ -4589,7 +4589,7 @@ body: |
successors: %bb.607(0x80000000)
%1628:vgpr_32 = PHI %1625, %bb.604, %1627, %bb.605
- SI_END_CF killed %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.607:
successors: %bb.608(0x80000000)
@@ -4634,7 +4634,7 @@ body: |
successors: %bb.613(0x80000000)
%1644:vgpr_32 = PHI %1641, %bb.610, %1643, %bb.611
- SI_END_CF killed %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.613:
successors: %bb.614(0x80000000)
@@ -4679,7 +4679,7 @@ body: |
successors: %bb.619(0x80000000)
%1660:vgpr_32 = PHI %1657, %bb.616, %1659, %bb.617
- SI_END_CF killed %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.619:
successors: %bb.620(0x80000000)
@@ -4724,7 +4724,7 @@ body: |
successors: %bb.625(0x80000000)
%1676:vgpr_32 = PHI %1673, %bb.622, %1675, %bb.623
- SI_END_CF killed %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.625:
successors: %bb.626(0x80000000)
@@ -4769,7 +4769,7 @@ body: |
successors: %bb.631(0x80000000)
%1692:vgpr_32 = PHI %1689, %bb.628, %1691, %bb.629
- SI_END_CF killed %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.631:
successors: %bb.632(0x80000000)
@@ -4814,7 +4814,7 @@ body: |
successors: %bb.637(0x80000000)
%1708:vgpr_32 = PHI %1705, %bb.634, %1707, %bb.635
- SI_END_CF killed %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.637:
successors: %bb.638(0x80000000)
@@ -4859,7 +4859,7 @@ body: |
successors: %bb.643(0x80000000)
%1724:vgpr_32 = PHI %1721, %bb.640, %1723, %bb.641
- SI_END_CF killed %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.643:
successors: %bb.644(0x80000000)
@@ -4904,7 +4904,7 @@ body: |
successors: %bb.649(0x80000000)
%1740:vgpr_32 = PHI %1737, %bb.646, %1739, %bb.647
- SI_END_CF killed %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.649:
successors: %bb.650(0x80000000)
@@ -4949,7 +4949,7 @@ body: |
successors: %bb.655(0x80000000)
%1756:vgpr_32 = PHI %1753, %bb.652, %1755, %bb.653
- SI_END_CF killed %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.655:
successors: %bb.656(0x80000000)
@@ -4994,7 +4994,7 @@ body: |
successors: %bb.661(0x80000000)
%1772:vgpr_32 = PHI %1769, %bb.658, %1771, %bb.659
- SI_END_CF killed %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.661:
successors: %bb.662(0x80000000)
@@ -5039,7 +5039,7 @@ body: |
successors: %bb.667(0x80000000)
%1788:vgpr_32 = PHI %1785, %bb.664, %1787, %bb.665
- SI_END_CF killed %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.667:
successors: %bb.668(0x80000000)
@@ -5084,7 +5084,7 @@ body: |
successors: %bb.673(0x80000000)
%1804:vgpr_32 = PHI %1801, %bb.670, %1803, %bb.671
- SI_END_CF killed %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.673:
successors: %bb.674(0x80000000)
@@ -5129,7 +5129,7 @@ body: |
successors: %bb.679(0x80000000)
%1820:vgpr_32 = PHI %1817, %bb.676, %1819, %bb.677
- SI_END_CF killed %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.679:
successors: %bb.680(0x80000000)
@@ -5174,7 +5174,7 @@ body: |
successors: %bb.685(0x80000000)
%1836:vgpr_32 = PHI %1833, %bb.682, %1835, %bb.683
- SI_END_CF killed %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.685:
successors: %bb.686(0x80000000)
@@ -5219,7 +5219,7 @@ body: |
successors: %bb.691(0x80000000)
%1852:vgpr_32 = PHI %1849, %bb.688, %1851, %bb.689
- SI_END_CF killed %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.691:
successors: %bb.692(0x80000000)
@@ -5264,7 +5264,7 @@ body: |
successors: %bb.697(0x80000000)
%1868:vgpr_32 = PHI %1865, %bb.694, %1867, %bb.695
- SI_END_CF killed %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.697:
successors: %bb.698(0x80000000)
@@ -5309,7 +5309,7 @@ body: |
successors: %bb.703(0x80000000)
%1884:vgpr_32 = PHI %1881, %bb.700, %1883, %bb.701
- SI_END_CF killed %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.703:
successors: %bb.704(0x80000000)
@@ -5354,7 +5354,7 @@ body: |
successors: %bb.709(0x80000000)
%1900:vgpr_32 = PHI %1897, %bb.706, %1899, %bb.707
- SI_END_CF killed %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.709:
successors: %bb.710(0x80000000)
@@ -5399,7 +5399,7 @@ body: |
successors: %bb.715(0x80000000)
%1916:vgpr_32 = PHI %1913, %bb.712, %1915, %bb.713
- SI_END_CF killed %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.715:
successors: %bb.716(0x80000000)
@@ -5444,7 +5444,7 @@ body: |
successors: %bb.721(0x80000000)
%1932:vgpr_32 = PHI %1929, %bb.718, %1931, %bb.719
- SI_END_CF killed %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.721:
successors: %bb.722(0x80000000)
@@ -5489,7 +5489,7 @@ body: |
successors: %bb.727(0x80000000)
%1948:vgpr_32 = PHI %1945, %bb.724, %1947, %bb.725
- SI_END_CF killed %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.727:
successors: %bb.728(0x80000000)
@@ -5534,7 +5534,7 @@ body: |
successors: %bb.733(0x80000000)
%1964:vgpr_32 = PHI %1961, %bb.730, %1963, %bb.731
- SI_END_CF killed %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.733:
successors: %bb.734(0x80000000)
@@ -5579,7 +5579,7 @@ body: |
successors: %bb.739(0x80000000)
%1980:vgpr_32 = PHI %1977, %bb.736, %1979, %bb.737
- SI_END_CF killed %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.739:
successors: %bb.740(0x80000000)
@@ -5624,7 +5624,7 @@ body: |
successors: %bb.745(0x80000000)
%1996:vgpr_32 = PHI %1993, %bb.742, %1995, %bb.743
- SI_END_CF killed %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.745:
successors: %bb.746(0x80000000)
@@ -5669,7 +5669,7 @@ body: |
successors: %bb.751(0x80000000)
%2012:vgpr_32 = PHI %2009, %bb.748, %2011, %bb.749
- SI_END_CF killed %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.751:
successors: %bb.752(0x80000000)
@@ -5714,7 +5714,7 @@ body: |
successors: %bb.757(0x80000000)
%2028:vgpr_32 = PHI %2025, %bb.754, %2027, %bb.755
- SI_END_CF killed %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.757:
successors: %bb.758(0x80000000)
@@ -5759,7 +5759,7 @@ body: |
successors: %bb.763(0x80000000)
%2044:vgpr_32 = PHI %2041, %bb.760, %2043, %bb.761
- SI_END_CF killed %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.763:
successors: %bb.764(0x80000000)
@@ -5804,7 +5804,7 @@ body: |
successors: %bb.769(0x80000000)
%2060:vgpr_32 = PHI %2057, %bb.766, %2059, %bb.767
- SI_END_CF killed %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.769:
@@ -6599,7 +6599,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7120,7 +7120,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7641,7 +7641,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8162,7 +8162,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8683,7 +8683,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9204,7 +9204,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9725,7 +9725,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10246,7 +10246,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10767,7 +10767,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11288,7 +11288,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11809,7 +11809,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12330,7 +12330,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12851,7 +12851,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13372,7 +13372,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13893,7 +13893,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14414,7 +14414,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14935,7 +14935,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15456,7 +15456,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15977,7 +15977,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16498,7 +16498,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17019,7 +17019,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17540,7 +17540,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18061,7 +18061,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18582,7 +18582,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19103,7 +19103,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19624,7 +19624,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20145,7 +20145,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20666,7 +20666,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -21187,7 +21187,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -21708,7 +21708,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -22229,7 +22229,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -22750,7 +22750,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -23271,7 +23271,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -23792,7 +23792,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24313,7 +24313,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24834,7 +24834,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -25355,7 +25355,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -25876,7 +25876,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26397,7 +26397,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26918,7 +26918,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -27439,7 +27439,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -27960,7 +27960,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -28481,7 +28481,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29002,7 +29002,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29523,7 +29523,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -30044,7 +30044,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -30565,7 +30565,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -31086,7 +31086,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -31607,7 +31607,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -32128,7 +32128,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -32649,7 +32649,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -33170,7 +33170,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -33691,7 +33691,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -34212,7 +34212,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -34733,7 +34733,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -35254,7 +35254,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -35775,7 +35775,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -36296,7 +36296,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -36817,7 +36817,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -37338,7 +37338,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -37859,7 +37859,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -38380,7 +38380,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -38901,7 +38901,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39422,7 +39422,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39943,7 +39943,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40464,7 +40464,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40985,7 +40985,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -41506,7 +41506,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -42027,7 +42027,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -42548,7 +42548,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -43069,7 +43069,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -43590,7 +43590,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -44111,7 +44111,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -44632,7 +44632,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -45153,7 +45153,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -45674,7 +45674,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -46195,7 +46195,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -46716,7 +46716,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -47237,7 +47237,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -47758,7 +47758,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -48279,7 +48279,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -48800,7 +48800,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -49321,7 +49321,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -49842,7 +49842,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -50363,7 +50363,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -50884,7 +50884,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -51405,7 +51405,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -51926,7 +51926,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -52447,7 +52447,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -52968,7 +52968,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53489,7 +53489,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -54010,7 +54010,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -54531,7 +54531,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -55052,7 +55052,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -55573,7 +55573,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56094,7 +56094,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56615,7 +56615,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -57136,7 +57136,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -57657,7 +57657,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -58178,7 +58178,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -58699,7 +58699,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -59220,7 +59220,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -59741,7 +59741,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -60262,7 +60262,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -60783,7 +60783,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -61304,7 +61304,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -61825,7 +61825,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -62346,7 +62346,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -62867,7 +62867,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -63388,7 +63388,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -63909,7 +63909,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -64430,7 +64430,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -64951,7 +64951,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -65472,7 +65472,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -65993,7 +65993,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -66514,7 +66514,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -67035,7 +67035,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -67556,7 +67556,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -68077,7 +68077,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -68598,7 +68598,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -69119,7 +69119,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -69640,7 +69640,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -70161,7 +70161,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -70682,7 +70682,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -71203,7 +71203,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -71724,7 +71724,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -72245,7 +72245,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -72763,7 +72763,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -75777,7 +75777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -76302,7 +76302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -76827,7 +76827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -77352,7 +77352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -77877,7 +77877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -78402,7 +78402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -78927,7 +78927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -79452,7 +79452,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -79977,7 +79977,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -80502,7 +80502,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -81027,7 +81027,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -81552,7 +81552,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -82077,7 +82077,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -82602,7 +82602,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -83127,7 +83127,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -83652,7 +83652,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -84177,7 +84177,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -84702,7 +84702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -85227,7 +85227,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -85752,7 +85752,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -86277,7 +86277,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -86802,7 +86802,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -87327,7 +87327,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -87852,7 +87852,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -88377,7 +88377,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -88902,7 +88902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -89427,7 +89427,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -89952,7 +89952,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -90477,7 +90477,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -91002,7 +91002,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -91527,7 +91527,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -92052,7 +92052,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -92577,7 +92577,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -93102,7 +93102,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -93627,7 +93627,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -94152,7 +94152,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -94677,7 +94677,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -95202,7 +95202,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -95727,7 +95727,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -96252,7 +96252,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -96777,7 +96777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -97302,7 +97302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -97827,7 +97827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -98352,7 +98352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -98877,7 +98877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -99402,7 +99402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -99927,7 +99927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -100452,7 +100452,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -100977,7 +100977,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -101502,7 +101502,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -102027,7 +102027,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -102552,7 +102552,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -103077,7 +103077,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -103602,7 +103602,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -104127,7 +104127,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -104652,7 +104652,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -105177,7 +105177,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -105702,7 +105702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -106227,7 +106227,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -106752,7 +106752,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -107277,7 +107277,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -107802,7 +107802,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -108327,7 +108327,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -108852,7 +108852,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -109377,7 +109377,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -109902,7 +109902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -110427,7 +110427,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -110952,7 +110952,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -111477,7 +111477,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -112002,7 +112002,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -112527,7 +112527,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -113052,7 +113052,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -113577,7 +113577,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -114102,7 +114102,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -114627,7 +114627,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -115152,7 +115152,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -115677,7 +115677,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -116202,7 +116202,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -116727,7 +116727,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -117252,7 +117252,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -117777,7 +117777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -118302,7 +118302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -118827,7 +118827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -119352,7 +119352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -119877,7 +119877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -120402,7 +120402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -120927,7 +120927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -121452,7 +121452,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -121977,7 +121977,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -122502,7 +122502,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -123027,7 +123027,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -123552,7 +123552,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -124077,7 +124077,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -124602,7 +124602,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -125127,7 +125127,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -125652,7 +125652,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -126177,7 +126177,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -126702,7 +126702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -127227,7 +127227,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -127752,7 +127752,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -128277,7 +128277,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -128802,7 +128802,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -129327,7 +129327,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -129852,7 +129852,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -130377,7 +130377,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -130902,7 +130902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -131427,7 +131427,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -131952,7 +131952,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -132477,7 +132477,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -133002,7 +133002,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -133527,7 +133527,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -134052,7 +134052,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -134577,7 +134577,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -135102,7 +135102,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -135627,7 +135627,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -136152,7 +136152,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -136677,7 +136677,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -137202,7 +137202,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -137727,7 +137727,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -138252,7 +138252,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -138777,7 +138777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -139302,7 +139302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -139827,7 +139827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -140352,7 +140352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -140877,7 +140877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -141402,7 +141402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -141927,7 +141927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -142449,7 +142449,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir
index 366cd4b7578442..9bfff07a7da504 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir
@@ -110,7 +110,7 @@ body: |
%28:vgpr_32 = PHI %29, %bb.6, %30, %bb.12
%31:vgpr_32 = PHI %32, %bb.6, %33, %bb.12
%34:vgpr_32 = PHI %35, %bb.6, %36, %bb.12
- SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.13
bb.3:
@@ -184,7 +184,7 @@ body: |
successors: %bb.10(0x40000000), %bb.8(0x40000000)
%85:vgpr_32 = PHI %24, %bb.4, %74, %bb.5
- SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%86:vgpr_32, %87:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %10, 24, 0, implicit $exec
%88:vgpr_32, dead $sgpr_null = V_ADDC_U32_e64 0, %9, %87, 0, implicit $exec
%89:vgpr_32, %90:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %15.sub0, 16, 0, implicit $exec
@@ -253,7 +253,7 @@ body: |
%76:vgpr_32 = PHI %98, %bb.8, %125, %bb.9
%140:vreg_64 = PHI %96, %bb.8, %115, %bb.9
%141:vreg_64 = PHI %93, %bb.8, %116, %bb.9
- SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%142:vgpr_32 = GLOBAL_LOAD_DWORD %62, 16, 0, implicit $exec :: (load (s32) from %ir.gep7, addrspace 1)
%77:vgpr_32 = V_ADD_U32_e64 %142, %139, 0, implicit $exec
GLOBAL_STORE_DWORD %140, %77, 0, 0, implicit $exec :: (store (s32) into %ir.phi4, addrspace 1)
@@ -270,7 +270,7 @@ body: |
bb.12:
successors: %bb.2(0x80000000)
- SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.2
bb.13:
@@ -1242,7 +1242,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2873,7 +2873,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4589,7 +4589,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4948,7 +4948,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.12": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6186,7 +6186,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7858,7 +7858,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9574,7 +9574,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9933,7 +9933,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.12": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir
index 7e9cf15988231f..99b8568c40791e 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir
@@ -92,7 +92,7 @@ body: |
bb.4:
%45:vreg_64 = PHI %8, %bb.1, %29, %bb.2
- SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%46:vgpr_32, %47:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %45.sub0, %1, 0, implicit $exec
%48:vgpr_32, dead $sgpr_null = V_ADDC_U32_e64 %45.sub1, %0, %47, 0, implicit $exec
%49:sreg_32_xm0 = V_READFIRSTLANE_B32 %46, implicit $exec
@@ -1363,7 +1363,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2857,7 +2857,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir
index 6fe28d7318b138..b06c255fb452d9 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir
@@ -196,7 +196,7 @@ body: |
bb.2:
successors: %bb.3(0x40000000), %bb.7(0x40000000)
- SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%112:vgpr_32 = GLOBAL_LOAD_USHORT %20, 0, 0, implicit $exec :: (load (s16) from %ir.p3, addrspace 1)
%113:vgpr_32 = GLOBAL_LOAD_USHORT %20, 2, 0, implicit $exec :: (load (s16) from %ir.p3 + 2, addrspace 1)
%114:vgpr_32 = V_LSHL_OR_B32_e64 %113, 16, %112, implicit $exec
@@ -225,7 +225,7 @@ body: |
%123:vgpr_32 = PHI %124, %bb.7, %125, %bb.11
%126:vgpr_32 = PHI %127, %bb.7, %128, %bb.11
- SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.13
bb.6:
@@ -278,7 +278,7 @@ body: |
bb.9:
successors: %bb.10(0x80000000)
- SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %19, %154, 0, 0, implicit $exec :: (store (s32) into %ir.p4, addrspace 1)
%162:vreg_64 = REG_SEQUENCE %158, %subreg.sub0, undef %163:vgpr_32, %subreg.sub1
%164:sreg_32 = S_MOV_B32 0
@@ -325,14 +325,14 @@ body: |
bb.11:
successors: %bb.5(0x80000000)
- SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%128:vgpr_32 = V_MUL_LO_U32_e64 %147, %143, implicit $exec
S_BRANCH %bb.5
bb.12:
successors: %bb.7(0x80000000)
- SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%146:vgpr_32 = V_MUL_LO_U32_e64 %114, %161, implicit $exec
%145:vgpr_32 = V_MUL_LO_U32_e64 %173, %195, implicit $exec
S_BRANCH %bb.7
@@ -4747,7 +4747,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5320,7 +5320,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6534,7 +6534,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.9": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7969,7 +7969,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.11": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8055,7 +8055,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.12": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13071,7 +13071,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13644,7 +13644,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14898,7 +14898,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.9": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16445,7 +16445,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.11": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16531,7 +16531,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.12": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir
index 356e533e402334..6ff0e51124d2b8 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir
@@ -51,7 +51,7 @@ body: |
%8:sreg_32 = PHI %9, %bb.3, %10, %bb.9
%11:vreg_64 = PHI undef %12:vreg_64, %bb.3, %13, %bb.9
%14:vreg_64 = PHI %15, %bb.3, undef %16:vreg_64, %bb.9
- SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%18:sreg_32 = S_AND_B32 %8, $exec_lo, implicit-def dead $scc
%19:sreg_32 = COPY %18
S_BRANCH %bb.8
@@ -86,7 +86,7 @@ body: |
%33:sreg_32 = PHI %34, %bb.6, %35, %bb.7
%36:vreg_64 = PHI %37, %bb.6, %38, %bb.7
- SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%40:sreg_32 = SI_IF %33, %bb.10, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
@@ -106,7 +106,7 @@ body: |
%34:sreg_32 = PHI %6, %bb.8, %28, %bb.2
%37:vreg_64 = PHI %47, %bb.8, undef %48:vreg_64, %bb.2
%49:vreg_64 = PHI %50, %bb.8, %27, %bb.2
- SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%39:sreg_32 = SI_IF %46, %bb.4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.7
@@ -154,7 +154,7 @@ body: |
bb.10:
%74:vreg_64 = PHI %36, %bb.4, %45, %bb.5
- SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%75:sreg_32_xm0 = V_READFIRSTLANE_B32 %74.sub0, implicit $exec
%76:sreg_32_xm0 = V_READFIRSTLANE_B32 %74.sub1, implicit $exec
$sgpr0 = COPY %75
@@ -455,7 +455,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1099,7 +1099,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1400,7 +1400,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2397,7 +2397,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2894,7 +2894,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3538,7 +3538,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3839,7 +3839,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4836,7 +4836,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir
index 7dfc0890aabf66..4b7d27a2038232 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir
@@ -105,7 +105,7 @@ body: |
successors: %bb.5(0x40000000), %bb.3(0x40000000)
%28:vgpr_32 = PHI %25, %bb.0, %27, %bb.1
- SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%29:vgpr_32 = DS_READ_U8_gfx9 %2, 0, 0, implicit $exec :: (load (s8) from %ir.p2, addrspace 3)
%30:vgpr_32 = DS_READ_U8_gfx9 %2, 1, 0, implicit $exec :: (load (s8) from %ir.p2 + 1, addrspace 3)
%31:vgpr_32 = DS_READ_U8_gfx9 %2, 2, 0, implicit $exec :: (load (s8) from %ir.p2 + 2, addrspace 3)
@@ -142,7 +142,7 @@ body: |
successors: %bb.12(0x40000000), %bb.7(0x40000000)
%46:vgpr_32 = PHI %38, %bb.3, %45, %bb.4
- SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%47:vgpr_32 = DS_READ_B32_gfx9 %2, 12, 0, implicit $exec :: (load (s32) from %ir.gep2, align 8, addrspace 3)
%48:vgpr_32 = V_ADD_U32_e64 %47, %46, 0, implicit $exec
%49:sreg_32 = S_XOR_B32 %7, %9, implicit-def dead $scc
@@ -219,11 +219,11 @@ body: |
successors: %bb.14(0x80000000)
%93:vgpr_32 = PHI %63, %bb.9, %71, %bb.10
- SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.14:
%94:vgpr_32 = PHI %52, %bb.7, %93, %bb.13
- SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%95:vgpr_32 = V_ADD_U32_e64 %36, %94, 0, implicit $exec
%96:sreg_32_xm0 = V_READFIRSTLANE_B32 %95, implicit $exec
$sgpr0 = COPY %96
@@ -998,7 +998,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1613,7 +1613,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2950,7 +2950,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2998,7 +2998,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3964,7 +3964,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4579,7 +4579,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5916,7 +5916,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5964,7 +5964,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir
index 5b4391c82a993a..48bb16f4139d6d 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir
@@ -112,7 +112,7 @@ body: |
%44:vgpr_32 = PHI undef %52:vgpr_32, %bb.1, %58, %bb.5
%41:vgpr_32 = PHI undef %52:vgpr_32, %bb.1, %59, %bb.5
%39:vgpr_32 = PHI undef %52:vgpr_32, %bb.1, %60, %bb.5
- SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%37:sreg_32 = SI_IF_BREAK %32, %36, implicit-def dead $scc
SI_LOOP %37, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.3
@@ -120,7 +120,7 @@ body: |
bb.3:
successors: %bb.4(0x40000000), %bb.6(0x40000000)
- SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%61:sreg_32 = SI_IF %28, %bb.6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
@@ -156,7 +156,7 @@ body: |
%68:vgpr_32 = PHI %56, %bb.3, %63, %bb.4
%69:vgpr_32 = PHI %54, %bb.3, %62, %bb.4
%70:vgpr_32 = PHI %51, %bb.3, %21, %bb.4
- SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%42:vgpr_32 = V_ADD_U32_e64 1, %40, 0, implicit $exec
%71:vgpr_32 = V_ADD_U32_e64 %68, %42, 0, implicit $exec
GLOBAL_STORE_DWORD %11, %71, 0, 0, implicit $exec :: (store (s32) into %ir.p1, addrspace 1)
@@ -167,7 +167,7 @@ body: |
S_BRANCH %bb.7
bb.7:
- SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%73:vgpr_32 = V_ADD3_U32_e64 %71, %43, %71, implicit $exec
%74:vgpr_32 = V_ADD3_U32_e64 %73, %68, %69, implicit $exec
%75:vgpr_32 = V_ADD3_U32_e64 %74, %70, %38, implicit $exec
@@ -1497,7 +1497,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1634,7 +1634,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.3": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2459,7 +2459,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2717,7 +2717,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.7": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4341,7 +4341,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4516,7 +4516,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.3": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5482,7 +5482,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5827,7 +5827,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.7": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir
index 0abdb905d0f2ee..7d4c4e3dc809a4 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir
@@ -111,7 +111,7 @@ body: |
%43:sreg_32 = PHI %44, %bb.5, %45, %bb.6
%46:vgpr_32 = PHI undef %47:vgpr_32, %bb.5, %48, %bb.6
- SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%50:sreg_32 = SI_IF %43, %bb.7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
@@ -128,7 +128,7 @@ body: |
%52:sreg_32 = PHI %27, %bb.1, %42, %bb.2
%44:sreg_32 = PHI %17, %bb.1, %39, %bb.2
%22:vreg_64 = PHI %20, %bb.1, %36, %bb.2
- SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%49:sreg_32 = SI_IF %52, %bb.3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -146,7 +146,7 @@ body: |
successors: %bb.8(0x04000000), %bb.1(0x7c000000)
%26:vgpr_32 = PHI %46, %bb.3, %51, %bb.4
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%24:sreg_32 = S_ADD_I32 %23, 1, implicit-def dead $scc
%56:sreg_32 = V_CMP_GE_U32_e64 %24, %0, implicit $exec
%19:sreg_32 = SI_IF_BREAK %56, %18, implicit-def dead $scc
@@ -154,7 +154,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %7, %26, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
S_ENDPGM 0
...
@@ -1306,7 +1306,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1556,7 +1556,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1843,7 +1843,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1991,7 +1991,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3291,7 +3291,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3575,7 +3575,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3902,7 +3902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4080,7 +4080,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir
index 6744895e4896f6..a27c271c3bfc39 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir
@@ -233,7 +233,7 @@ body: |
%128:vgpr_32 = PHI %71, %bb.1, %129, %bb.33
%130:vgpr_32 = PHI %73, %bb.1, %131, %bb.33
%132:vgpr_32 = PHI undef %120:vgpr_32, %bb.1, %133, %bb.33
- SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%59:sreg_32 = SI_IF_BREAK %48, %58, implicit-def dead $scc
%134:sreg_32 = S_ANDN2_B32 %43, $exec_lo, implicit-def dead $scc
%135:sreg_32 = S_AND_B32 %57, $exec_lo, implicit-def dead $scc
@@ -271,7 +271,7 @@ body: |
%166:vgpr_32 = PHI undef %162:vgpr_32, %bb.7, %167, %bb.9
%168:vgpr_32 = PHI undef %162:vgpr_32, %bb.7, %147, %bb.9
%169:vgpr_32 = PHI undef %170:vgpr_32, %bb.7, %167, %bb.9
- SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%172:sreg_32 = S_ORN2_B32 %156, $exec_lo, implicit-def dead $scc
S_BRANCH %bb.8
@@ -400,7 +400,7 @@ body: |
%274:vgpr_32 = PHI undef %266:vgpr_32, %bb.10, %275, %bb.15
%276:vgpr_32 = PHI undef %277:vgpr_32, %bb.10, %248, %bb.15
%278:vgpr_32 = PHI %236, %bb.10, %279, %bb.15
- SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%280:sreg_32 = S_MOV_B32 0
%281:sreg_32 = SI_IF %254, %bb.17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.16
@@ -444,7 +444,7 @@ body: |
%273:vgpr_32 = PHI %71, %bb.11, undef %297:vgpr_32, %bb.13
%275:vgpr_32 = PHI %73, %bb.11, undef %297:vgpr_32, %bb.13
%279:vgpr_32 = PHI undef %298:vgpr_32, %bb.11, %282, %bb.13
- SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%263:sreg_32 = COPY $exec_lo
%261:sreg_32 = S_XOR_B32 $exec_lo, -1, implicit-def dead $scc
%299:sreg_32 = S_ANDN2_B32 %239, $exec_lo, implicit-def dead $scc
@@ -495,7 +495,7 @@ body: |
%344:vgpr_32 = PHI %270, %bb.12, %345, %bb.19
%346:vgpr_32 = PHI %272, %bb.12, %347, %bb.19
%348:vgpr_32 = PHI %274, %bb.12, %349, %bb.19
- SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%350:sreg_32 = SI_IF %316, %bb.33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.20
@@ -539,7 +539,7 @@ body: |
%345:vgpr_32 = PHI %385, %bb.23, %69, %bb.22
%347:vgpr_32 = PHI %386, %bb.23, %71, %bb.22
%349:vgpr_32 = PHI %387, %bb.23, %73, %bb.22
- SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%389:sreg_32 = S_AND_B32 %373, $exec_lo, implicit-def dead $scc
%325:sreg_32 = COPY %389
%390:sreg_32 = S_ANDN2_B32 %262, $exec_lo, implicit-def dead $scc
@@ -579,7 +579,7 @@ body: |
%413:vgpr_32 = PHI undef %411:vgpr_32, %bb.18, %414, %bb.24
%415:vgpr_32 = PHI undef %411:vgpr_32, %bb.18, %69, %bb.24
%416:vgpr_32 = PHI undef %417:vgpr_32, %bb.18, %414, %bb.24
- SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%418:sreg_32 = S_AND_B32 %408, $exec_lo, implicit-def dead $scc
%419:sreg_32 = COPY %418
%420:sreg_32 = S_ANDN2_B32 %262, $exec_lo, implicit-def dead $scc
@@ -716,19 +716,19 @@ body: |
successors: %bb.27(0x80000000)
%456:vgpr_32 = PHI %119, %bb.36, %484, %bb.29
- SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.27
bb.31:
successors: %bb.25(0x80000000)
%437:vgpr_32 = PHI %454, %bb.27, %473, %bb.28
- SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.25
bb.32:
%488:vgpr_32 = PHI %435, %bb.25, %451, %bb.26
- SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%489:vgpr_32 = V_ADD3_U32_e64 %30, %488, 100, implicit $exec
%490:sreg_32_xm0 = V_READFIRSTLANE_B32 %489, implicit $exec
$sgpr0 = COPY %490
@@ -753,7 +753,7 @@ body: |
%129:vgpr_32 = PHI %346, %bb.17, %71, %bb.20
%131:vgpr_32 = PHI %348, %bb.17, %73, %bb.20
%133:vgpr_32 = PHI %276, %bb.17, %112, %bb.20
- SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%495:sreg_32 = S_ANDN2_B32 %90, $exec_lo, implicit-def dead $scc
%496:sreg_32 = S_AND_B32 %494, $exec_lo, implicit-def dead $scc
%110:sreg_32 = S_OR_B32 %495, %496, implicit-def dead $scc
@@ -771,7 +771,7 @@ body: |
bb.34:
successors: %bb.35(0x40000000), %bb.25(0x40000000)
- SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%503:sreg_32 = S_XOR_B32 %39, -1, implicit-def dead $scc
%504:sreg_32 = S_XOR_B32 %42, -1, implicit-def dead $scc
%443:sreg_32 = SI_IF %503, %bb.25, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -802,7 +802,7 @@ body: |
%247:vgpr_32 = PHI %209, %bb.8, %147, %bb.6
%250:vgpr_32 = PHI %210, %bb.8, %149, %bb.6
%249:vgpr_32 = PHI %211, %bb.8, %151, %bb.6
- SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%507:sreg_32 = S_XOR_B32 %506, -1, implicit-def dead $scc
%144:sreg_32 = SI_IF_BREAK %505, %143, implicit-def dead $scc
%508:sreg_32 = S_ANDN2_B32 %140, $exec_lo, implicit-def dead $scc
@@ -814,7 +814,7 @@ body: |
bb.38:
successors: %bb.14(0x40000000), %bb.10(0x40000000)
- SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%234:sreg_32 = S_MOV_B32 0
%244:sreg_32 = SI_IF %142, %bb.10, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.14
@@ -4975,7 +4975,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6431,7 +6431,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12331,7 +12331,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14092,7 +14092,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16469,7 +16469,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18452,7 +18452,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20204,7 +20204,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24327,7 +24327,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24411,7 +24411,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24486,7 +24486,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -25296,7 +25296,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26005,7 +26005,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.34": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26846,7 +26846,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -27282,7 +27282,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.38": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -32423,7 +32423,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -34153,7 +34153,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40308,7 +40308,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -42189,7 +42189,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -44829,7 +44829,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -47086,7 +47086,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -49129,7 +49129,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53604,7 +53604,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53688,7 +53688,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53763,7 +53763,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -54733,7 +54733,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -55582,7 +55582,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.34": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56475,7 +56475,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56967,7 +56967,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.38": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir
index f35dff5b8eab88..c7150e147d76ea 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir
@@ -180,7 +180,7 @@ body: |
%81:vgpr_32 = PHI %82, %bb.11, %83, %bb.14
%84:vgpr_32 = PHI %85, %bb.11, %86, %bb.14
- SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.15
bb.6:
@@ -218,7 +218,7 @@ body: |
bb.10:
successors: %bb.14(0x04000000), %bb.6(0x7c000000)
- SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%101:sreg_32 = S_ADD_I32 %95, 1, implicit-def dead $scc
%93:sreg_32 = S_ADD_I32 %92, %34, implicit-def dead $scc
%91:sreg_32 = S_ADD_I32 %90, %65, implicit-def dead $scc
@@ -254,13 +254,13 @@ body: |
bb.13:
successors: %bb.11(0x80000000)
- SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.11
bb.14:
successors: %bb.5(0x80000000)
- SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.15:
@@ -328,7 +328,7 @@ body: |
%164:vgpr_32 = PHI %139, %bb.15, %163, %bb.16
%165:vgpr_32 = PHI %131, %bb.15, %153, %bb.16
- SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%66:sreg_32 = S_ADD_I32 %65, 1, implicit-def dead $scc
%64:sreg_32 = S_ADD_I32 %63, 10, implicit-def dead $scc
%166:sreg_32 = V_CMP_GE_U32_e64 %66, %3, implicit $exec
@@ -339,7 +339,7 @@ body: |
bb.18:
successors: %bb.19(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%35:sreg_32 = S_ADD_I32 %34, 1, implicit-def dead $scc
%167:vgpr_32 = V_ADD_U32_e64 %35, %49, 0, implicit $exec
%168:vgpr_32 = V_ASHRREV_I32_e64 31, %165, implicit $exec
@@ -357,7 +357,7 @@ body: |
S_BRANCH %bb.19
bb.19:
- SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%176:vgpr_32 = V_ADD_U32_e64 %164, %49, 0, implicit $exec
%177:sreg_32_xm0 = V_READFIRSTLANE_B32 %176, implicit $exec
$sgpr0 = COPY %177
@@ -3233,7 +3233,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3997,7 +3997,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.10": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4986,7 +4986,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.13": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5071,7 +5071,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7161,7 +7161,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7386,7 +7386,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.18": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7835,7 +7835,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.19": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11087,7 +11087,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11976,7 +11976,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.10": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13145,7 +13145,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.13": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13242,7 +13242,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15895,7 +15895,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16207,7 +16207,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.18": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16883,7 +16883,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.19": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir
index c5a44b5d7c854a..08a866634750d6 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir
@@ -175,7 +175,7 @@ body: |
bb.7:
successors: %bb.8(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%22:sreg_32 = S_ADD_I32 %21, 1, implicit-def dead $scc
%83:sreg_32 = V_CMP_GE_U32_e64 %22, %1, implicit $exec
%20:sreg_32 = SI_IF_BREAK %83, %19, implicit-def dead $scc
@@ -184,7 +184,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%85:vgpr_32 = V_ADD3_U32_e64 %23, %84, %17, implicit $exec
%86:vgpr_32 = V_ADD3_U32_e64 %85, %26, 100, implicit $exec
%87:sreg_32_xm0 = V_READFIRSTLANE_B32 %86, implicit $exec
@@ -2972,7 +2972,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.7": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3154,7 +3154,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6448,7 +6448,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.7": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6685,7 +6685,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir
index 465454e5aeaf10..913c04284b8975 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir
@@ -199,7 +199,7 @@ body: |
%38:vgpr_32 = PHI %95, %bb.4, %106, %bb.9
%107:vgpr_32 = PHI %97, %bb.4, %108, %bb.9
%109:vgpr_32 = PHI %99, %bb.4, %110, %bb.9
- SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.10
bb.7:
@@ -228,7 +228,7 @@ body: |
bb.9:
successors: %bb.6(0x80000000)
- SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%106:vgpr_32 = V_MUL_LO_U32_e64 %110, %117, implicit $exec
%108:vgpr_32 = COPY %102, implicit $exec
S_BRANCH %bb.6
@@ -290,7 +290,7 @@ body: |
%68:vgpr_32 = PHI %107, %bb.10, %69, %bb.11
%162:vgpr_32 = PHI %137, %bb.10, %161, %bb.11
%163:vgpr_32 = PHI %129, %bb.10, %151, %bb.11
- SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%92:sreg_32 = S_ADD_I32 %91, 1, implicit-def dead $scc
%164:sreg_32 = V_CMP_GE_U32_e64 %92, %2, implicit $exec
%90:sreg_32 = SI_IF_BREAK %164, %89, implicit-def dead $scc
@@ -300,7 +300,7 @@ body: |
bb.13:
successors: %bb.14(0x04000000), %bb.2(0x7c000000)
- SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%66:sreg_32 = S_ADD_I32 %65, 1, implicit-def dead $scc
%165:sreg_32 = V_CMP_GE_U32_e64 %66, %3, implicit $exec
%64:sreg_32 = SI_IF_BREAK %165, %63, implicit-def dead $scc
@@ -310,7 +310,7 @@ body: |
bb.14:
successors: %bb.15(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%36:sreg_32 = S_ADD_I32 %35, 1, implicit-def dead $scc
%166:vgpr_32 = V_ADD_U32_e64 %36, %52, 0, implicit $exec
%167:vgpr_32 = V_ASHRREV_I32_e64 31, %163, implicit $exec
@@ -327,7 +327,7 @@ body: |
S_BRANCH %bb.15
bb.15:
- SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%175:vgpr_32 = V_ADD_U32_e64 %162, %52, 0, implicit $exec
%176:sreg_32_xm0 = V_READFIRSTLANE_B32 %175, implicit $exec
$sgpr0 = COPY %176
@@ -4266,7 +4266,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5010,7 +5010,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.9": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7043,7 +7043,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7245,7 +7245,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.13": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7429,7 +7429,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7841,7 +7841,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.15": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12325,7 +12325,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13212,7 +13212,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.9": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15787,7 +15787,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16085,7 +16085,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.13": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16359,7 +16359,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17017,7 +17017,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.15": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir
index 20932f9c0331dc..00ea6dd596bf53 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir
@@ -154,7 +154,7 @@ body: |
%46:sreg_32 = PHI undef %70:sreg_32, %bb.1, %71, %bb.14
%72:vgpr_32 = PHI undef %73:vgpr_32, %bb.1, %74, %bb.14
%75:vgpr_32 = PHI undef %73:vgpr_32, %bb.1, %76, %bb.14
- SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%44:sreg_32 = SI_IF_BREAK %36, %43, implicit-def dead $scc
%77:vgpr_32 = COPY %45, implicit $exec
%78:sreg_32 = S_ANDN2_B32 %31, $exec_lo, implicit-def dead $scc
@@ -195,7 +195,7 @@ body: |
%90:sreg_32 = PHI %96, %bb.4, %102, %bb.17
%94:sreg_32 = PHI undef %103:sreg_32, %bb.4, %104, %bb.17
%74:vgpr_32 = PHI %95, %bb.4, %25, %bb.17
- SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%105:sreg_32 = S_XOR_B32 %90, -1, implicit-def dead $scc
%92:sreg_32 = SI_IF_BREAK %87, %91, implicit-def dead $scc
%106:sreg_32 = S_ANDN2_B32 %82, $exec_lo, implicit-def dead $scc
@@ -248,12 +248,12 @@ body: |
successors: %bb.9(0x80000000)
%119:vgpr_32 = PHI %75, %bb.16, %72, %bb.11
- SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.9
bb.13:
%126:vgpr_32 = PHI %117, %bb.9, %124, %bb.10
- SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%127:vgpr_32 = V_ADD3_U32_e64 %24, %126, 100, implicit $exec
%128:sreg_32_xm0 = V_READFIRSTLANE_B32 %127, implicit $exec
$sgpr0 = COPY %128
@@ -266,7 +266,7 @@ body: |
%131:sreg_32 = PHI %130, %bb.18, %115, %bb.8
%71:sreg_32 = PHI undef %132:sreg_32, %bb.18, %112, %bb.8
%76:vgpr_32 = PHI undef %133:vgpr_32, %bb.18, %113, %bb.8
- SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%135:sreg_32 = S_ANDN2_B32 %61, $exec_lo, implicit-def dead $scc
%136:sreg_32 = S_AND_B32 %131, $exec_lo, implicit-def dead $scc
%69:sreg_32 = S_OR_B32 %135, %136, implicit-def dead $scc
@@ -280,7 +280,7 @@ body: |
bb.15:
successors: %bb.16(0x40000000), %bb.9(0x40000000)
- SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%140:sreg_32 = S_XOR_B32 %30, -1, implicit-def dead $scc
%123:sreg_32 = SI_IF %140, %bb.9, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.16
@@ -297,7 +297,7 @@ body: |
%141:sreg_32 = PHI %99, %bb.5, %111, %bb.7
%142:sreg_32 = PHI %99, %bb.5, %110, %bb.7
%104:sreg_32 = PHI undef %143:sreg_32, %bb.5, %108, %bb.7
- SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%144:sreg_32 = S_ANDN2_B32 %96, $exec_lo, implicit-def dead $scc
%145:sreg_32 = S_AND_B32 %142, $exec_lo, implicit-def dead $scc
%102:sreg_32 = S_OR_B32 %144, %145, implicit-def dead $scc
@@ -309,7 +309,7 @@ body: |
bb.18:
successors: %bb.8(0x40000000), %bb.14(0x40000000)
- SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%130:sreg_32 = S_MOV_B32 -1
%134:sreg_32 = SI_IF %84, %bb.14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.8
@@ -2294,7 +2294,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3351,7 +3351,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4336,7 +4336,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4411,7 +4411,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4647,7 +4647,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5003,7 +5003,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.15": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5296,7 +5296,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5622,7 +5622,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.18": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7972,7 +7972,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9219,7 +9219,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10359,7 +10359,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10434,7 +10434,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10702,7 +10702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11138,7 +11138,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.15": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11449,7 +11449,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11823,7 +11823,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.18": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir
index fbcf447271740c..778c7c137eaaef 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir
@@ -115,7 +115,7 @@ body: |
bb.2:
successors: %bb.3(0x80000000)
- SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%51:vgpr_32 = V_ADD_U32_e64 -2, %50, 0, implicit $exec
%52:vgpr_32 = V_MUL_LO_U32_e64 %30, %51, implicit $exec
%53:vgpr_32 = V_MUL_LO_U32_e64 100, %27, implicit $exec
@@ -136,7 +136,7 @@ body: |
S_BRANCH %bb.4
bb.4:
- SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %8, %59, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%62:vgpr_32 = V_ADD3_U32_e64 %59, %61, %29, implicit $exec
%63:vgpr_32 = V_ADD3_U32_e64 %62, %53, %41, implicit $exec
@@ -1774,7 +1774,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2205,7 +2205,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.4": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4192,7 +4192,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4635,7 +4635,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.4": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir
index 4338457e42221f..820eb937c53e37 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir
@@ -63,7 +63,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
DS_WRITE_B32_gfx9 %1, %8, 0, 0, implicit $exec :: (store (s32) into %ir.p, addrspace 3)
S_ENDPGM 0
...
@@ -618,7 +618,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1273,7 +1273,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir
index a7713d7b1d91fc..d139ec03fe6115 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir
@@ -97,7 +97,7 @@ body: |
bb.4:
%45:vreg_64 = PHI %8, %bb.1, %29, %bb.2
- SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%46:vgpr_32, %47:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %45.sub0, %1, 0, implicit $exec
%48:vgpr_32, dead $sgpr_null = V_ADDC_U32_e64 %45.sub1, %0, %47, 0, implicit $exec
%49:sreg_32_xm0 = V_READFIRSTLANE_B32 %46, implicit $exec
@@ -1368,7 +1368,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2862,7 +2862,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir
index 69b4a4ba60ee96..9381db82edbb29 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir
@@ -121,7 +121,7 @@ body: |
%34:vgpr_32 = PHI %54, %bb.2, %57, %bb.3
%65:vgpr_32 = PHI %46, %bb.2, %27, %bb.3
%38:vgpr_32 = PHI %48, %bb.2, %56, %bb.3
- SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%32:sreg_32 = SI_IF_BREAK %64, %31, implicit-def dead $scc
SI_LOOP %32, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -129,7 +129,7 @@ body: |
bb.6:
successors: %bb.7(0x80000000)
- SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%66:vgpr_32 = V_MUL_LO_U32_e64 %38, %27, implicit $exec
GLOBAL_STORE_DWORD %16, %66, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%67:sreg_32 = S_MOV_B32 0
@@ -148,7 +148,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%75:vgpr_32 = V_ADD_U32_e64 %72, %74, 0, implicit $exec
%76:vgpr_32 = V_ADD_U32_e64 %75, %38, 0, implicit $exec
%77:vgpr_32 = V_ADD3_U32_e64 %76, %66, %75, implicit $exec
@@ -2276,7 +2276,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2421,7 +2421,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.6": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2815,7 +2815,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5371,7 +5371,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5492,7 +5492,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.6": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5880,7 +5880,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir
index 97206f60d7adf8..e1cf748656c422 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir
@@ -70,7 +70,7 @@ body: |
successors: %bb.5(0x40000000), %bb.3(0x40000000)
%25:vgpr_32 = PHI %22, %bb.0, %24, %bb.1
- SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%26:vgpr_32 = DS_READ_U8_gfx9 %2, 0, 0, implicit $exec :: (load (s8) from %ir.p2, addrspace 3)
%27:vgpr_32 = DS_READ_U8_gfx9 %2, 1, 0, implicit $exec :: (load (s8) from %ir.p2 + 1, addrspace 3)
%28:vgpr_32 = DS_READ_U8_gfx9 %2, 2, 0, implicit $exec :: (load (s8) from %ir.p2 + 2, addrspace 3)
@@ -108,7 +108,7 @@ body: |
successors: %bb.12(0x40000000), %bb.7(0x40000000)
%44:vgpr_32 = PHI %34, %bb.3, %43, %bb.4
- SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%45:vgpr_32 = V_LSHL_OR_B32_e64 %40, 16, %39, implicit $exec
%46:vgpr_32 = DS_READ_B32_gfx9 %2, 12, 0, implicit $exec :: (load (s32) from %ir.gep2, align 8, addrspace 3)
%47:vgpr_32 = V_ADD_U32_e64 %46, %44, 0, implicit $exec
@@ -186,11 +186,11 @@ body: |
successors: %bb.14(0x80000000)
%92:vgpr_32 = PHI %62, %bb.9, %70, %bb.10
- SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.14:
%93:vgpr_32 = PHI %51, %bb.7, %92, %bb.13
- SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%94:vgpr_32 = V_ADD3_U32_e64 %32, %93, 100, implicit $exec
%95:sreg_32_xm0 = V_READFIRSTLANE_B32 %94, implicit $exec
$sgpr0 = COPY %95
@@ -899,7 +899,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1596,7 +1596,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2953,7 +2953,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3001,7 +3001,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3902,7 +3902,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4599,7 +4599,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5956,7 +5956,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6004,7 +6004,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir
index f2c85920feb3bf..7e03a9f5d7052c 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir
@@ -97,7 +97,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%52:vreg_64 = REG_SEQUENCE %23, %subreg.sub0, undef %53:vgpr_32, %subreg.sub1
%54:vreg_64, $sgpr_null = V_MAD_U64_U32_e64 %16, %33, %52, 0, implicit $exec
GLOBAL_STORE_DWORD %11, %54.sub0, 0, 0, implicit $exec :: (store (s32) into %ir.p5, addrspace 1)
@@ -1712,7 +1712,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3453,7 +3453,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir
index 434b3ec6c24bbc..0b5cb308ca1f24 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir
@@ -102,7 +102,7 @@ body: |
successors: %bb.5(0x80000000)
%44:vgpr_32 = PHI %39, %bb.1, %43, %bb.2
- SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%45:vreg_64 = REG_SEQUENCE %9, %subreg.sub0, %8, %subreg.sub1
%46:vreg_64 = REG_SEQUENCE %11, %subreg.sub0, %10, %subreg.sub1
%47:vreg_64 = REG_SEQUENCE %13, %subreg.sub0, %12, %subreg.sub1
@@ -181,7 +181,7 @@ body: |
%64:vgpr_32 = PHI %78, %bb.7, %106, %bb.8
%107:vgpr_32 = PHI %94, %bb.7, %105, %bb.8
%88:vgpr_32 = PHI %94, %bb.7, %85, %bb.8
- SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %47, %88, 0, 0, implicit $exec :: (store (s32) into %ir.p9, addrspace 1)
%108:vgpr_32 = GLOBAL_LOAD_DWORD %46, 0, 0, implicit $exec :: (load (s32) from %ir.p10, addrspace 1)
%109:vgpr_32 = V_MUL_LO_U32_e64 %107, %64, implicit $exec
@@ -196,7 +196,7 @@ body: |
bb.10:
successors: %bb.11(0x04000000), %bb.6(0x7c000000)
- SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%86:vgpr_32 = V_ADD_U32_e64 1, %85, 0, implicit $exec
%112:vgpr_32 = GLOBAL_LOAD_DWORD %45, 0, 0, implicit $exec :: (load (s32) from %ir.p11, addrspace 1)
%113:vgpr_32 = V_ADD_U32_e64 %86, %107, 0, implicit $exec
@@ -210,7 +210,7 @@ body: |
bb.11:
successors: %bb.12(0x80000000)
- SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%115:vreg_64 = REG_SEQUENCE %64, %subreg.sub0, undef %116:vgpr_32, %subreg.sub1
%117:vreg_64, $sgpr_null = V_MAD_U64_U32_e64 %64, %107, %115, 0, implicit $exec
%118:vgpr_32 = COPY %117.sub0
@@ -248,7 +248,7 @@ body: |
bb.13:
successors: %bb.14(0x04000000), %bb.5(0x7c000000)
- SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%141:vgpr_32 = V_ADD_U32_e64 %135, %1, 0, implicit $exec
GLOBAL_STORE_DWORD %50, %141, 0, 0, implicit $exec :: (store (s32) into %ir.p5, addrspace 1)
%62:vgpr_32 = V_ADD_U32_e64 1, %61, 0, implicit $exec
@@ -261,7 +261,7 @@ body: |
bb.14:
successors: %bb.15(0x40000000), %bb.16(0x40000000)
- SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%143:vgpr_32 = V_MUL_LO_U32_e64 100, %44, implicit $exec
%144:sreg_32 = V_CMP_GT_U32_e64 %143, %141, implicit $exec
%145:sreg_32 = SI_IF %144, %bb.16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -276,7 +276,7 @@ body: |
successors: %bb.17(0x80000000)
%146:vgpr_32 = PHI %143, %bb.14, %141, %bb.15
- SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%147:sreg_32 = S_MOV_B32 0
bb.17:
@@ -291,7 +291,7 @@ body: |
S_BRANCH %bb.18
bb.18:
- SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%153:vgpr_32 = V_ADD_U32_e64 100, %37, 0, implicit $exec
%154:vgpr_32 = V_MUL_LO_U32_e64 %143, %150, implicit $exec
%155:vgpr_32 = V_MUL_LO_U32_e64 %153, %154, implicit $exec
@@ -2356,7 +2356,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5241,7 +5241,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5866,7 +5866,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.10": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6413,7 +6413,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.11": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8038,7 +8038,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.13": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8447,7 +8447,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8629,7 +8629,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8859,7 +8859,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.18": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11228,7 +11228,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13991,7 +13991,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14508,7 +14508,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.10": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14950,7 +14950,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.11": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16112,7 +16112,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.13": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16372,7 +16372,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16554,7 +16554,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16778,7 +16778,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.18": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir
index 231fcdea8cfdec..26896997f786e7 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir
@@ -104,7 +104,7 @@ body: |
bb.2:
successors: %bb.3(0x80000000)
- SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%58:vgpr_32 = V_ADD_U32_e64 -2, %57, 0, implicit $exec
%59:vgpr_32 = V_MUL_LO_U32_e64 %37, %58, implicit $exec
%60:vgpr_32 = V_MUL_LO_U32_e64 100, %34, implicit $exec
@@ -125,7 +125,7 @@ body: |
S_BRANCH %bb.4
bb.4:
- SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %10, %67, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%69:vgpr_32 = V_ADD_U32_e64 %67, %66, 0, implicit $exec
%70:vgpr_32 = V_ADD3_U32_e64 %69, %36, %60, implicit $exec
@@ -2157,7 +2157,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2604,7 +2604,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.4": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4793,7 +4793,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5232,7 +5232,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.4": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir
index 2d3b9bf2c0c1e5..6103b28c345f42 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir
@@ -87,7 +87,7 @@ body: |
bb.4:
%39:vreg_64 = PHI %7, %bb.1, %21, %bb.2
- SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%40:sreg_32_xm0 = V_READFIRSTLANE_B32 %39.sub0, implicit $exec
%41:sreg_32_xm0 = V_READFIRSTLANE_B32 %39.sub1, implicit $exec
$sgpr0 = COPY %40
@@ -1046,7 +1046,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2169,7 +2169,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir
index cb14edadc7a0ef..7407fd3d7542b1 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir
@@ -144,7 +144,7 @@ body: |
%76:vgpr_32 = PHI %34, %bb.1, %49, %bb.2
%77:vgpr_32 = PHI %40, %bb.1, %50, %bb.2
%78:vgpr_32 = PHI %37, %bb.1, %48, %bb.2
- SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%79:vreg_64 = REG_SEQUENCE %25, %subreg.sub0, undef %80:vgpr_32, %subreg.sub1
%81:vreg_64, $sgpr_null = V_MAD_U64_U32_e64 %75, %27, %79, 0, implicit $exec
%82:vreg_64 = REG_SEQUENCE %76, %subreg.sub0, undef %83:vgpr_32, %subreg.sub1
@@ -410,7 +410,7 @@ body: |
bb.8:
successors: %bb.11(0x40000000), %bb.9(0x40000000)
- SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%321:sreg_32 = V_CMP_GT_U32_e64 %27, %304, implicit $exec
%322:sreg_32 = S_MOV_B32 0
%323:sreg_32 = SI_IF %321, %bb.9, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -450,7 +450,7 @@ body: |
successors: %bb.13(0x40000000), %bb.14(0x40000000)
%341:sreg_32 = PHI %324, %bb.9, %337, %bb.10
- SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%342:vreg_64 = REG_SEQUENCE %1, %subreg.sub0, %0, %subreg.sub1
%343:vreg_64 = REG_SEQUENCE %3, %subreg.sub0, %2, %subreg.sub1
%344:sreg_32 = SI_IF %341, %bb.14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -463,7 +463,7 @@ body: |
GLOBAL_STORE_DWORD %20, %345, 0, 0, implicit $exec :: (store (s32) into %ir.p1, addrspace 1)
bb.14:
- SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%346:vgpr_32 = GLOBAL_LOAD_UBYTE %342, 0, 0, implicit $exec :: (load (s8) from %ir.p10, addrspace 1)
%347:vgpr_32 = GLOBAL_LOAD_UBYTE %342, 1, 0, implicit $exec :: (load (s8) from %ir.p10 + 1, addrspace 1)
%348:vgpr_32 = V_LSHL_OR_B32_e64 %347, 8, %346, implicit $exec
@@ -3743,7 +3743,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18969,7 +18969,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19748,7 +19748,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19940,7 +19940,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -23810,7 +23810,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39036,7 +39036,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39815,7 +39815,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40007,7 +40007,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir
index 772ac742919337..f4cfb571a117a7 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir
@@ -401,7 +401,7 @@ body: |
%332:vgpr_32 = PHI %58, %bb.0, %104, %bb.1
%333:vgpr_32 = PHI %65, %bb.0, %325, %bb.1
%334:vgpr_32 = PHI %81, %bb.0, %106, %bb.1
- SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%335:sreg_32 = V_CMP_GT_U32_e64 %34, %333, implicit $exec
%336:sreg_32 = S_MOV_B32 0
%337:sreg_32 = SI_IF %335, %bb.3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -439,7 +439,7 @@ body: |
successors: %bb.7(0x40000000), %bb.8(0x40000000)
%351:sreg_32 = PHI %338, %bb.3, %348, %bb.4
- SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%352:vgpr_32 = V_LSHL_OR_B32_e64 %345, 16, %344, implicit $exec
%353:sreg_32 = SI_IF %351, %bb.8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.7
@@ -451,7 +451,7 @@ body: |
GLOBAL_STORE_DWORD %29, %354, 0, 0, implicit $exec :: (store (s32) into %ir.p1, addrspace 1)
bb.8:
- SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%355:vgpr_32 = V_ADD_U32_e64 %352, %82, 0, implicit $exec
GLOBAL_STORE_DWORD %28, %355, 0, 0, implicit $exec :: (store (s32) into %ir.p9, addrspace 1)
S_ENDPGM 0
@@ -19821,7 +19821,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20540,7 +20540,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20708,7 +20708,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40513,7 +40513,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -41232,7 +41232,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -41400,7 +41400,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir
index 324976cec20e57..a7b06da340f3b0 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir
@@ -100,7 +100,7 @@ body: |
successors: %bb.3(0x40000000), %bb.14(0x40000000)
%43:vgpr_32 = PHI %40, %bb.0, %42, %bb.1
- SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%44:sreg_32 = SI_IF %25, %bb.14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.3
@@ -291,7 +291,7 @@ body: |
%216:vgpr_32 = PHI %217, %bb.14, %218, %bb.13
%219:vgpr_32 = PHI %220, %bb.14, %204, %bb.13
%221:vgpr_32 = PHI %222, %bb.14, %223, %bb.13
- SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%225:vreg_64 = REG_SEQUENCE %2, %subreg.sub0, %1, %subreg.sub1
%226:vreg_64 = REG_SEQUENCE %4, %subreg.sub0, %3, %subreg.sub1
S_BRANCH %bb.17
@@ -300,7 +300,7 @@ body: |
successors: %bb.12(0x40000000), %bb.10(0x40000000)
%227:vgpr_32 = PHI %147, %bb.5, %169, %bb.6
- SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%228:vgpr_32 = V_ADD_U32_e64 100, %102, 0, implicit $exec
%229:vgpr_32 = COPY %155.sub1
%230:vgpr_32 = COPY %155.sub2
@@ -407,7 +407,7 @@ body: |
%315:vgpr_32 = PHI %231, %bb.10, %264, %bb.11
%316:vgpr_32 = PHI %232, %bb.10, %264, %bb.11
%317:vgpr_32 = PHI %136, %bb.10, %264, %bb.11
- SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%223:vgpr_32 = V_ADD_U32_e64 %308, %227, 0, implicit $exec
%215:vgpr_32 = V_SUB_U32_e64 %43, %318, 0, implicit $exec
%319:vgpr_32 = V_ADD_U32_e64 %223, %215, 0, implicit $exec
@@ -476,7 +476,7 @@ body: |
bb.16:
successors: %bb.14(0x80000000)
- SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%364:vgpr_32 = V_ADD_U32_e64 %48, %363, 0, implicit $exec
%365:vgpr_32 = V_ADD_U32_e64 %52, %363, 0, implicit $exec
%366:sreg_64 = SI_PC_ADD_REL_OFFSET target-flags(amdgpu-gotprel32-lo) @array7, target-flags(amdgpu-gotprel32-hi) @array7, implicit-def dead $scc
@@ -2178,7 +2178,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10889,7 +10889,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11076,7 +11076,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15259,7 +15259,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17558,7 +17558,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.16": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20530,7 +20530,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29241,7 +29241,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29428,7 +29428,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -33611,7 +33611,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -35904,7 +35904,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.16": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir
index f8e354de9aa0ca..2b3f3de5d5a238 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir
@@ -163,7 +163,7 @@ body: |
%109:vgpr_32 = PHI %31, %bb.0, %24, %bb.1
%110:vgpr_32 = PHI %39, %bb.0, %1, %bb.1
%111:vgpr_32 = PHI %35, %bb.0, %72, %bb.1
- SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%112:vgpr_32 = V_ADD_U32_e64 %97, %56, 0, implicit $exec
%113:vgpr_32 = V_MUL_LO_U32_e64 %112, %49, implicit $exec
%114:vgpr_32 = V_SUB_U32_e64 %113, %39, 0, implicit $exec
@@ -4296,7 +4296,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9224,7 +9224,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir
index 0d595ede9c9921..e740457b46c6f3 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir
@@ -88,7 +88,7 @@ body: |
%43:sreg_32 = PHI %44, %bb.5, %45, %bb.6
%46:vgpr_32 = PHI undef %47:vgpr_32, %bb.5, %48, %bb.6
- SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%50:sreg_32 = SI_IF %43, %bb.7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
@@ -105,7 +105,7 @@ body: |
%52:sreg_32 = PHI %27, %bb.1, %42, %bb.2
%44:sreg_32 = PHI %17, %bb.1, %39, %bb.2
%22:vreg_64 = PHI %20, %bb.1, %36, %bb.2
- SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%49:sreg_32 = SI_IF %52, %bb.3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -123,7 +123,7 @@ body: |
successors: %bb.8(0x04000000), %bb.1(0x7c000000)
%26:vgpr_32 = PHI %46, %bb.3, %51, %bb.4
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%24:sreg_32 = S_ADD_I32 %23, 1, implicit-def dead $scc
%56:sreg_32 = V_CMP_GE_U32_e64 %24, %0, implicit $exec
%19:sreg_32 = SI_IF_BREAK %56, %18, implicit-def dead $scc
@@ -131,7 +131,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %7, %26, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
S_ENDPGM 0
...
@@ -1312,7 +1312,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1596,7 +1596,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1923,7 +1923,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2101,7 +2101,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3371,7 +3371,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3621,7 +3621,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3908,7 +3908,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4056,7 +4056,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir
index eb605289042d51..93338b3e5abe26 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir
@@ -71,7 +71,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%27:vgpr_32 = V_ASHRREV_I32_e64 31, %25, implicit $exec
%28:vreg_64 = REG_SEQUENCE %25, %subreg.sub0, %27, %subreg.sub1
%29:vreg_64 = nsw V_LSHLREV_B64_pseudo_e64 2, %28, implicit $exec
@@ -933,7 +933,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2733,7 +2733,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir
index 904a7971321ea2..abd655422748f3 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir
@@ -141,7 +141,7 @@ body: |
bb.4:
successors: %bb.5(0x04000000), %bb.2(0x7c000000)
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%65:vgpr_32 = V_ADD_U32_e64 %59, %64, 0, implicit $exec
%37:vgpr_32 = V_MUL_LO_U32_e64 %29, %65, implicit $exec
%35:sreg_32 = S_ADD_I32 %34, 2, implicit-def dead $scc
@@ -153,7 +153,7 @@ body: |
bb.5:
successors: %bb.6(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%30:vgpr_32 = V_SUB_U32_e64 %44, %64, 0, implicit $exec
%28:sreg_32 = S_ADD_I32 %27, 1, implicit-def dead $scc
%67:sreg_32 = V_CMP_GE_U32_e64 %28, %1, implicit $exec
@@ -162,7 +162,7 @@ body: |
S_BRANCH %bb.6
bb.6:
- SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%68:vgpr_32 = V_MAX_U32_e64 1, %1, implicit $exec
%69:vgpr_32 = V_ADD_U32_e64 %68, %23, 0, implicit $exec
%70:vgpr_32 = V_ADD3_U32_e64 %36, %29, %65, implicit $exec
@@ -2339,7 +2339,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.4": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2598,7 +2598,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.5": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2799,7 +2799,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.6": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5393,7 +5393,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.4": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5698,7 +5698,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.5": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5949,7 +5949,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.6": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir
index d25842ccbf55b6..5413abbe8c1ee2 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir
@@ -140,7 +140,7 @@ body: |
%28:vgpr_32 = PHI %40, %bb.2, %52, %bb.3
%60:vgpr_32 = PHI %42, %bb.2, %21, %bb.3
%32:vgpr_32 = PHI %44, %bb.2, %51, %bb.3
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%26:sreg_32 = SI_IF_BREAK %59, %25, implicit-def dead $scc
SI_LOOP %26, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -148,7 +148,7 @@ body: |
bb.6:
successors: %bb.7(0x80000000)
- SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%61:vgpr_32 = V_MUL_LO_U32_e64 %32, %31, implicit $exec
GLOBAL_STORE_DWORD %10, %61, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%62:sreg_32 = S_MOV_B32 0
@@ -167,7 +167,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%70:vgpr_32 = V_ADD3_U32_e64 %67, %69, %32, implicit $exec
%71:vgpr_32 = V_ADD3_U32_e64 %70, %61, %60, implicit $exec
%72:vgpr_32 = V_ADD3_U32_e64 %21, %71, 100, implicit $exec
@@ -1931,7 +1931,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2044,7 +2044,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.6": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2400,7 +2400,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4488,7 +4488,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4625,7 +4625,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.6": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4993,7 +4993,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir b/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir
index 5eb8c608708de9..0800fb3fc9fd84 100644
--- a/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir
+++ b/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir
@@ -29,7 +29,7 @@ body: |
successors: %bb.1, %bb.2
liveins: $sgpr0
- $exec_lo = S_MOV_B32 -1
+ $exec_lo = bb-prolog S_MOV_B32 -1
renamable $vgpr0 = V_MBCNT_LO_U32_B32_e64 -1, 0, implicit $exec
renamable $sgpr0 = S_BFE_U32 killed renamable $sgpr0, 589836, implicit-def dead $scc
renamable $vcc_lo = V_CMP_GT_U32_e64 killed $sgpr0, killed $vgpr0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll b/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll
index 7694e30a612af3..5f4dd2a8b850c2 100644
--- a/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll
+++ b/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll
@@ -27,10 +27,10 @@ define i32 @prolog_spill(i32 %arg0, i32 %arg1, i32 %arg2) {
; REGALLOC-NEXT: bb.1.Flow:
; REGALLOC-NEXT: successors: %bb.2(0x40000000), %bb.4(0x40000000)
; REGALLOC-NEXT: {{ $}}
- ; REGALLOC-NEXT: $vgpr63 = SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
- ; REGALLOC-NEXT: $sgpr4 = SI_RESTORE_S32_FROM_VGPR $vgpr63, 0, implicit-def $sgpr4_sgpr5
- ; REGALLOC-NEXT: $sgpr5 = SI_RESTORE_S32_FROM_VGPR $vgpr63, 1
- ; REGALLOC-NEXT: renamable $sgpr4_sgpr5 = S_OR_SAVEEXEC_B64 killed renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; REGALLOC-NEXT: $vgpr63 = bb-prolog SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
+ ; REGALLOC-NEXT: $sgpr4 = bb-prolog SI_RESTORE_S32_FROM_VGPR $vgpr63, 0, implicit-def $sgpr4_sgpr5
+ ; REGALLOC-NEXT: $sgpr5 = bb-prolog SI_RESTORE_S32_FROM_VGPR $vgpr63, 1
+ ; REGALLOC-NEXT: renamable $sgpr4_sgpr5 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def dead $scc, implicit $exec
; REGALLOC-NEXT: $vgpr0 = SI_SPILL_V32_RESTORE %stack.6, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.6, addrspace 5)
; REGALLOC-NEXT: SI_SPILL_V32_SAVE killed $vgpr0, %stack.5, $sgpr32, 0, implicit $exec :: (store (s32) into %stack.5, addrspace 5)
; REGALLOC-NEXT: renamable $sgpr4_sgpr5 = S_AND_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
@@ -60,10 +60,10 @@ define i32 @prolog_spill(i32 %arg0, i32 %arg1, i32 %arg2) {
; REGALLOC-NEXT: S_BRANCH %bb.1
; REGALLOC-NEXT: {{ $}}
; REGALLOC-NEXT: bb.4.bb.3:
- ; REGALLOC-NEXT: $vgpr63 = SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
- ; REGALLOC-NEXT: $sgpr4 = SI_RESTORE_S32_FROM_VGPR $vgpr63, 2, implicit-def $sgpr4_sgpr5
- ; REGALLOC-NEXT: $sgpr5 = SI_RESTORE_S32_FROM_VGPR killed $vgpr63, 3
- ; REGALLOC-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; REGALLOC-NEXT: $vgpr63 = bb-prolog SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
+ ; REGALLOC-NEXT: $sgpr4 = bb-prolog SI_RESTORE_S32_FROM_VGPR $vgpr63, 2, implicit-def $sgpr4_sgpr5
+ ; REGALLOC-NEXT: $sgpr5 = bb-prolog SI_RESTORE_S32_FROM_VGPR killed $vgpr63, 3
+ ; REGALLOC-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; REGALLOC-NEXT: $vgpr0 = SI_SPILL_V32_RESTORE %stack.5, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.5, addrspace 5)
; REGALLOC-NEXT: renamable $vgpr0 = V_LSHL_ADD_U32_e64 killed $vgpr0, 2, $vgpr0, implicit $exec
; REGALLOC-NEXT: SI_RETURN implicit killed $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir b/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir
index 7ecb1cd7a53438..e66cb4cf4ab7a0 100644
--- a/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir
@@ -61,7 +61,7 @@ body: |
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.7(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 killed [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 killed [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY4:%[0-9]+]]:vgpr_32 = COPY killed [[COPY2]]
; CHECK-NEXT: [[S_AND_B32_1:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[S_OR_SAVEEXEC_B32_]], implicit-def $scc
; CHECK-NEXT: $exec_lo = S_XOR_B32_term $exec_lo, [[S_AND_B32_1]], implicit-def $scc
@@ -75,7 +75,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.5
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.7:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed [[S_AND_B32_1]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed [[S_AND_B32_1]], implicit-def $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.2(0x40000000), %bb.5(0x40000000)
@@ -128,7 +128,7 @@ body: |
S_BRANCH %bb.5
bb.7:
- SI_END_CF %14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll b/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll
index 5752fde325d0cb..075196617ee212 100644
--- a/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll
+++ b/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll
@@ -133,7 +133,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.9(0x40000000), %bb.10(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr18_sgpr19, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr18_sgpr19, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr18_sgpr19, implicit-def $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr40_sgpr41, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr12_sgpr13 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.10, implicit $exec
@@ -150,7 +150,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.11(0x40000000), %bb.12(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr46_sgpr47, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr12_sgpr13 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.12, implicit $exec
@@ -167,7 +167,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.13(0x40000000), %bb.14(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr44_sgpr45, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr12_sgpr13 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.14, implicit $exec
@@ -184,7 +184,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.15(0x40000000), %bb.16(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr42_sgpr43, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr42_sgpr43, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr36_sgpr37 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.16, implicit $exec
@@ -208,7 +208,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.17(0x40000000), %bb.18(0x40000000)
; GFX90A-NEXT: liveins: $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr36_sgpr37, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr36_sgpr37, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr66_sgpr67, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.18, implicit $exec
@@ -225,7 +225,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.19(0x40000000), %bb.20(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr64_sgpr65, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.20, implicit $exec
@@ -242,7 +242,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.21(0x40000000), %bb.22(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr54_sgpr55, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.22, implicit $exec
@@ -259,7 +259,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.23(0x40000000), %bb.24(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr52_sgpr53, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.24, implicit $exec
@@ -276,7 +276,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.25(0x40000000), %bb.26(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr50_sgpr51, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.26, implicit $exec
@@ -293,7 +293,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.27(0x40000000), %bb.28(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr48_sgpr49, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.28, implicit $exec
@@ -310,7 +310,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.34(0x40000000), %bb.29(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr38_sgpr39, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECNZ %bb.34, implicit $exec
@@ -319,7 +319,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.30(0x40000000), %bb.31(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GFX90A-NEXT: dead renamable $sgpr4_sgpr5 = S_AND_B64 killed renamable $sgpr34_sgpr35, $exec, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr4 = S_CSELECT_B32 1, 0, implicit killed $scc
; GFX90A-NEXT: S_CMP_LG_U32 killed renamable $sgpr4, 1, implicit-def $scc
@@ -404,7 +404,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.6(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr24_sgpr25, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr24_sgpr25, implicit-def $scc
; GFX90A-NEXT: S_BRANCH %bb.6
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.37.bb27:
@@ -443,7 +443,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.36(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr38_sgpr39, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr38_sgpr39, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr38_sgpr39 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr48_sgpr49 = S_AND_B64 killed renamable $sgpr42_sgpr43, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr50_sgpr51 = S_AND_B64 killed renamable $sgpr66_sgpr67, $exec, implicit-def dead $scc
@@ -495,7 +495,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.38(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr68_sgpr69, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr40_sgpr41, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr40_sgpr41, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr42_sgpr43 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr66_sgpr67 = S_AND_B64 killed renamable $sgpr44_sgpr45, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr52_sgpr53 = S_AND_B64 killed renamable $sgpr68_sgpr69, $exec, implicit-def dead $scc
@@ -545,7 +545,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.40(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr42_sgpr43, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr42_sgpr43, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr44_sgpr45 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr68_sgpr69 = S_AND_B64 killed renamable $sgpr46_sgpr47, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr54_sgpr55 = S_AND_B64 killed renamable $sgpr54_sgpr55, $exec, implicit-def dead $scc
@@ -643,7 +643,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.42(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr58_sgpr59, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $sgpr70_sgpr71, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr44_sgpr45, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr44_sgpr45, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr46_sgpr47 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr54_sgpr55 = S_AND_B64 killed renamable $sgpr62_sgpr63, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr64_sgpr65 = S_AND_B64 killed renamable $sgpr64_sgpr65, $exec, implicit-def dead $scc
@@ -758,7 +758,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.45(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr62_sgpr63, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr62_sgpr63, implicit-def $scc
; GFX90A-NEXT: S_BRANCH %bb.45
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.56.bb90:
@@ -862,7 +862,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.61(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr54_sgpr55, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr54_sgpr55, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr54_sgpr55 = S_MOV_B64 0
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.61.Flow30:
@@ -908,7 +908,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: $sgpr12_sgpr13 = S_ANDN2_SAVEEXEC_B64 $sgpr12_sgpr13, implicit-def $exec, implicit-def $scc, implicit $exec
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def $scc
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.66.Flow14:
; GFX90A-NEXT: successors: %bb.8(0x80000000)
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir b/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir
index 4b5b94d359c639..68b3fdd1c54303 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir
@@ -53,7 +53,7 @@ body: |
bb.4:
successors: %bb.5
- $exec = S_OR_B64 $exec, %20, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %20, implicit-def $scc
bb.5:
successors: %bb.7, %bb.6
@@ -90,7 +90,7 @@ body: |
bb.10:
successors: %bb.9
- $exec = S_OR_B64 $exec, %29, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %29, implicit-def $scc
%36:vgpr_32 = COPY %28
%37:sreg_64_xexec = V_CMP_NE_U32_e64 0, %36, implicit $exec
%38:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, %37, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir
index 58994a9acf9aa7..ead8128439f3d7 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir
@@ -88,7 +88,7 @@ body: |
bb.10:
successors: %bb.12(0x80000000)
- $exec = S_OR_B64 $exec, killed %22, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %22, implicit-def $scc
S_BRANCH %bb.12
bb.11:
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir
index 88717d21a05bed..c19569053e1057 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir
@@ -91,7 +91,7 @@ body: |
bb.5:
successors: %bb.9(0x80000000)
- $exec = S_OR_B64 $exec, %46, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %46, implicit-def $scc
%47:vgpr_32 = COPY killed %48
%49:vgpr_32 = COPY killed %50
%51:vreg_128 = COPY killed %52
@@ -111,7 +111,7 @@ body: |
bb.6:
successors: %bb.7(0x40000000), %bb.13(0x40000000)
- $exec = S_OR_B64 $exec, killed %60, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %60, implicit-def $scc
%61:sreg_64 = V_CMP_NE_U32_e64 0, killed %62, implicit $exec
%63:vreg_128 = COPY killed %64
%65:vgpr_32 = COPY killed %66
@@ -140,7 +140,7 @@ body: |
bb.9:
successors: %bb.6(0x04000000), %bb.4(0x7c000000)
- $exec = S_OR_B64 $exec, %45, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %45, implicit-def $scc
%62:vgpr_32 = COPY killed %42
%66:vgpr_32 = COPY killed %41
%76:vgpr_32 = COPY killed %40
@@ -182,7 +182,7 @@ body: |
bb.12:
successors: %bb.15(0x40000000), %bb.5(0x40000000)
- $exec = S_OR_B64 $exec, killed %85, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %85, implicit-def $scc
%87:sreg_64 = V_CMP_LT_U32_e64 11, killed %28, implicit $exec
%88:vgpr_32 = V_MOV_B32_e32 -1, implicit $exec
%56:sreg_64 = COPY %24
@@ -199,7 +199,7 @@ body: |
bb.13:
successors: %bb.14(0x40000000), %bb.16(0x40000000)
- $exec = S_OR_B64 $exec, killed %67, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %67, implicit-def $scc
%91:vgpr_32 = COPY killed %65
%92:vreg_128 = COPY killed %63
%93:sreg_64 = V_CMP_NE_U32_e64 0, killed %91, implicit $exec
@@ -232,7 +232,7 @@ body: |
bb.16:
successors: %bb.3(0x80000000)
- $exec = S_OR_B64 $exec, killed %95, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %95, implicit-def $scc
%103:vreg_128 = COPY killed %94
%104:vreg_128 = COPY killed %103
%5:vreg_128 = COPY killed %104
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir b/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir
index 3beb6b6f02d36a..bd2a4469021bb8 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir
@@ -84,7 +84,7 @@ body: |
bb.7:
successors: %bb.6(0x40000000), %bb.8(0x40000000)
- $exec = S_OR_B64 $exec, %24, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %24, implicit-def $scc
%25:vreg_128 = COPY killed %21
%26:vreg_128 = COPY killed %20
%27:sreg_64 = COPY killed %19
@@ -97,7 +97,7 @@ body: |
bb.8:
successors: %bb.5(0x80000000)
- $exec = S_OR_B64 $exec, killed %28, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %28, implicit-def $scc
%29:vreg_128 = COPY killed %26
%1:vreg_128 = COPY killed %29
S_BRANCH %bb.5
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir b/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir
index 481befdb0e1877..83cf664af32339 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir
@@ -90,7 +90,7 @@ body: |
bb.7:
successors: %bb.6(0x80000000)
- $exec = S_OR_B64 $exec, killed %23, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %23, implicit-def $scc
%30:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
%15:vreg_128 = COPY %13
%16:vgpr_32 = COPY killed %30
@@ -157,7 +157,7 @@ body: |
bb.16:
successors: %bb.17(0x80000000)
- $exec = S_OR_B64 $exec, killed %50, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %50, implicit-def $scc
S_BRANCH %bb.17
bb.17:
diff --git a/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir b/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir
index 25b2105bacd2c1..85aac1fd627b84 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir
@@ -89,7 +89,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- $exec = S_OR_B64 $exec, killed %1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %1, implicit-def $scc
%33:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
undef %32.sub0:vreg_128 = COPY killed %31.sub0
%32.sub2:vreg_128 = COPY %33
diff --git a/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll b/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll
index dbed069faef1aa..3f8aa97b637bd2 100644
--- a/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll
+++ b/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll
@@ -351,10 +351,9 @@ define void @recursive_phis(i1 %cond, ptr addrspace(5) %ptr) {
; GISEL-ASM: ; %bb.0: ; %entry
; GISEL-ASM-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GISEL-ASM-NEXT: v_and_b32_e32 v0, 1, v0
+; GISEL-ASM-NEXT: s_lshr_b32 s6, s32, 6
; GISEL-ASM-NEXT: s_mov_b64 s[4:5], exec
; GISEL-ASM-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; GISEL-ASM-NEXT: s_lshr_b32 s6, s32, 6
-; GISEL-ASM-NEXT: s_xor_b64 s[4:5], vcc, s[4:5]
; GISEL-ASM-NEXT: v_mov_b32_e32 v0, s6
; GISEL-ASM-NEXT: s_and_saveexec_b64 s[6:7], vcc
; GISEL-ASM-NEXT: ; %bb.1: ; %then
@@ -362,19 +361,20 @@ define void @recursive_phis(i1 %cond, ptr addrspace(5) %ptr) {
; GISEL-ASM-NEXT: ; %bb.2: ; %finallyendcf.split
; GISEL-ASM-NEXT: s_or_b64 exec, exec, s[6:7]
; GISEL-ASM-NEXT: s_mov_b64 s[8:9], src_private_base
-; GISEL-ASM-NEXT: s_mov_b64 s[6:7], 0
+; GISEL-ASM-NEXT: s_xor_b64 s[6:7], vcc, s[4:5]
+; GISEL-ASM-NEXT: s_mov_b64 s[4:5], 0
; GISEL-ASM-NEXT: v_mov_b32_e32 v1, s9
; GISEL-ASM-NEXT: v_mov_b32_e32 v2, 7
; GISEL-ASM-NEXT: .LBB11_3: ; %finally
; GISEL-ASM-NEXT: ; =>This Inner Loop Header: Depth=1
-; GISEL-ASM-NEXT: s_and_b64 s[8:9], exec, s[4:5]
-; GISEL-ASM-NEXT: s_or_b64 s[6:7], s[8:9], s[6:7]
+; GISEL-ASM-NEXT: s_and_b64 s[8:9], exec, s[6:7]
+; GISEL-ASM-NEXT: s_or_b64 s[4:5], s[8:9], s[4:5]
; GISEL-ASM-NEXT: flat_store_dword v[0:1], v2
; GISEL-ASM-NEXT: s_waitcnt vmcnt(0)
-; GISEL-ASM-NEXT: s_andn2_b64 exec, exec, s[6:7]
+; GISEL-ASM-NEXT: s_andn2_b64 exec, exec, s[4:5]
; GISEL-ASM-NEXT: s_cbranch_execnz .LBB11_3
; GISEL-ASM-NEXT: ; %bb.4: ; %end
-; GISEL-ASM-NEXT: s_or_b64 exec, exec, s[6:7]
+; GISEL-ASM-NEXT: s_or_b64 exec, exec, s[4:5]
; GISEL-ASM-NEXT: s_waitcnt lgkmcnt(0)
; GISEL-ASM-NEXT: s_setpc_b64 s[30:31]
entry:
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir b/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir
index b7f36c5dcd542f..a0b00585e359ce 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir
@@ -26,10 +26,10 @@ body: |
; GXN-NEXT: bb.1:
; GXN-NEXT: successors: %bb.2(0x80000000)
; GXN-NEXT: {{ $}}
- ; GXN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def $scc
+ ; GXN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def $scc
; GXN-NEXT: {{ $}}
; GXN-NEXT: bb.2:
- ; GXN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GXN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GXN-NEXT: [[DEF:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; GXN-NEXT: [[DEF1:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; GXN-NEXT: DS_WRITE_B32 [[DEF]], [[DEF1]], 0, 0, implicit $m0, implicit $exec :: (store (s32), addrspace 3)
@@ -43,10 +43,10 @@ body: |
%2:sgpr_64 = COPY $exec
bb.1:
- $exec = S_OR_B64 $exec, %1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %1, implicit-def $scc
bb.2:
- $exec = S_OR_B64 $exec, %2, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %2, implicit-def $scc
%5:vgpr_32 = IMPLICIT_DEF
%6:vgpr_32 = IMPLICIT_DEF
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll b/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll
index 873d28b54337b7..6fa995f4ff282c 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll
@@ -69,6 +69,7 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB0_4
; GCN-O0-NEXT: ; %bb.1: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -76,14 +77,12 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s2, 0xf000
; GCN-O0-NEXT: s_mov_b32 s0, 0
; GCN-O0-NEXT: ; kill: def $sgpr0 killed $sgpr0 def $sgpr0_sgpr1
; GCN-O0-NEXT: s_mov_b32 s1, s2
; GCN-O0-NEXT: ; kill: def $sgpr4_sgpr5 killed $sgpr4_sgpr5 def $sgpr4_sgpr5_sgpr6_sgpr7
; GCN-O0-NEXT: s_mov_b64 s[6:7], s[0:1]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v0
; GCN-O0-NEXT: v_mov_b32_e32 v1, v0
; GCN-O0-NEXT: v_mov_b32_e32 v2, v3
@@ -102,6 +101,8 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB0_3
; GCN-O0-NEXT: ; %bb.2: ; %bb.inner.then
+; GCN-O0-NEXT: s_waitcnt expcnt(1)
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -109,9 +110,7 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -240,6 +239,7 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB1_3
; GCN-O0-NEXT: ; %bb.1: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -247,14 +247,12 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s2, 0xf000
; GCN-O0-NEXT: s_mov_b32 s0, 0
; GCN-O0-NEXT: ; kill: def $sgpr0 killed $sgpr0 def $sgpr0_sgpr1
; GCN-O0-NEXT: s_mov_b32 s1, s2
; GCN-O0-NEXT: ; kill: def $sgpr4_sgpr5 killed $sgpr4_sgpr5 def $sgpr4_sgpr5_sgpr6_sgpr7
; GCN-O0-NEXT: s_mov_b64 s[6:7], s[0:1]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v0
; GCN-O0-NEXT: v_mov_b32_e32 v1, v0
; GCN-O0-NEXT: v_mov_b32_e32 v2, v3
@@ -273,6 +271,8 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB1_4
; GCN-O0-NEXT: ; %bb.2: ; %bb.inner.then
+; GCN-O0-NEXT: s_waitcnt expcnt(1)
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -280,9 +280,7 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -316,9 +314,9 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: v_readlane_b32 s2, v4, 4
; GCN-O0-NEXT: v_readlane_b32 s3, v4, 5
; GCN-O0-NEXT: s_or_b64 exec, exec, s[2:3]
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 2
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
@@ -456,17 +454,18 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB2_6
; GCN-O0-NEXT: ; %bb.1: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[6:7]
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s0, 2
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: v_cmp_ne_u32_e64 s[0:1], v0, s0
; GCN-O0-NEXT: s_mov_b64 s[2:3], exec
; GCN-O0-NEXT: s_and_b64 s[0:1], s[2:3], s[0:1]
; GCN-O0-NEXT: s_xor_b64 s[2:3], s[0:1], s[2:3]
+; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_writelane_b32 v4, s2, 4
; GCN-O0-NEXT: v_writelane_b32 v4, s3, 5
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
@@ -493,6 +492,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_xor_b64 exec, exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB2_5
; GCN-O0-NEXT: ; %bb.3: ; %bb.then
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -500,9 +500,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -518,6 +516,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: buffer_store_dword v0, v[1:2], s[0:3], 0 addr64
; GCN-O0-NEXT: s_branch .LBB2_5
; GCN-O0-NEXT: .LBB2_4: ; %bb.else
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -525,9 +524,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 2
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -725,13 +722,13 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_xor_b64 exec, exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB3_8
; GCN-O0-NEXT: ; %bb.2: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v6, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[8:9]
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s0, 0xf000
; GCN-O0-NEXT: s_mov_b32 s2, 0
; GCN-O0-NEXT: s_mov_b32 s4, s2
@@ -741,11 +738,12 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: ; kill: def $sgpr0_sgpr1 killed $sgpr0_sgpr1 def $sgpr0_sgpr1_sgpr2_sgpr3
; GCN-O0-NEXT: s_mov_b64 s[2:3], s[4:5]
; GCN-O0-NEXT: v_mov_b32_e32 v1, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: buffer_store_dword v1, v[2:3], s[0:3], 0 addr64 offset:4
; GCN-O0-NEXT: s_mov_b32 s0, 2
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[2:3], v0, s0
; GCN-O0-NEXT: s_mov_b64 s[0:1], exec
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: v_writelane_b32 v6, s0, 4
; GCN-O0-NEXT: v_writelane_b32 v6, s1, 5
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
@@ -771,13 +769,13 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: buffer_store_dword v0, v[1:2], s[0:3], 0 addr64 offset:8
; GCN-O0-NEXT: s_branch .LBB3_7
; GCN-O0-NEXT: .LBB3_4: ; %bb.outer.else
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v6, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[8:9]
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s1, 0xf000
; GCN-O0-NEXT: s_mov_b32 s0, 0
; GCN-O0-NEXT: s_mov_b32 s2, s0
@@ -787,10 +785,11 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: ; kill: def $sgpr4_sgpr5 killed $sgpr4_sgpr5 def $sgpr4_sgpr5_sgpr6_sgpr7
; GCN-O0-NEXT: s_mov_b64 s[6:7], s[2:3]
; GCN-O0-NEXT: v_mov_b32_e32 v1, 3
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: buffer_store_dword v1, v[2:3], s[4:7], 0 addr64 offset:12
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[2:3], v0, s0
; GCN-O0-NEXT: s_mov_b64 s[0:1], exec
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: v_writelane_b32 v6, s0, 6
; GCN-O0-NEXT: v_writelane_b32 v6, s1, 7
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
@@ -930,6 +929,7 @@ define amdgpu_kernel void @s_endpgm_unsafe_barrier(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB4_2
; GCN-O0-NEXT: ; %bb.1: ; %bb.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -937,14 +937,12 @@ define amdgpu_kernel void @s_endpgm_unsafe_barrier(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v3, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v3, 1
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s2, 0xf000
; GCN-O0-NEXT: s_mov_b32 s4, 0
; GCN-O0-NEXT: ; kill: def $sgpr4 killed $sgpr4 def $sgpr4_sgpr5
; GCN-O0-NEXT: s_mov_b32 s5, s2
; GCN-O0-NEXT: ; kill: def $sgpr0_sgpr1 killed $sgpr0_sgpr1 def $sgpr0_sgpr1_sgpr2_sgpr3
; GCN-O0-NEXT: s_mov_b64 s[2:3], s[4:5]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_ashrrev_i32_e64 v2, 31, v0
; GCN-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GCN-O0-NEXT: v_mov_b32_e32 v1, v2
@@ -1071,6 +1069,8 @@ define void @scc_liveness(i32 %arg) local_unnamed_addr #0 {
; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: .LBB5_1: ; %bb1
; GCN-O0-NEXT: ; =>This Inner Loop Header: Depth=1
+; GCN-O0-NEXT: s_waitcnt expcnt(1)
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
@@ -1082,9 +1082,7 @@ define void @scc_liveness(i32 %arg) local_unnamed_addr #0 {
; GCN-O0-NEXT: v_readlane_b32 s7, v7, 1
; GCN-O0-NEXT: v_writelane_b32 v7, s6, 4
; GCN-O0-NEXT: v_writelane_b32 v7, s7, 5
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s4, 0x207
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_cmp_lt_i32_e64 s[4:5], v0, s4
; GCN-O0-NEXT: s_or_b64 s[4:5], s[4:5], s[8:9]
; GCN-O0-NEXT: v_writelane_b32 v7, s4, 6
@@ -1280,14 +1278,14 @@ define void @scc_liveness(i32 %arg) local_unnamed_addr #0 {
; GCN-O0-NEXT: v_readlane_b32 s8, v7, 16
; GCN-O0-NEXT: v_readlane_b32 s9, v7, 17
; GCN-O0-NEXT: s_or_b64 exec, exec, s[8:9]
-; GCN-O0-NEXT: v_readlane_b32 s6, v7, 4
-; GCN-O0-NEXT: v_readlane_b32 s7, v7, 5
-; GCN-O0-NEXT: v_readlane_b32 s4, v7, 14
-; GCN-O0-NEXT: v_readlane_b32 s5, v7, 15
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:52 ; 4-byte Folded Reload
+; GCN-O0-NEXT: v_readlane_b32 s6, v7, 4
+; GCN-O0-NEXT: v_readlane_b32 s7, v7, 5
+; GCN-O0-NEXT: v_readlane_b32 s4, v7, 14
+; GCN-O0-NEXT: v_readlane_b32 s5, v7, 15
; GCN-O0-NEXT: s_and_b64 s[4:5], exec, s[4:5]
; GCN-O0-NEXT: s_or_b64 s[4:5], s[4:5], s[6:7]
; GCN-O0-NEXT: s_mov_b64 s[6:7], 0
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir b/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir
index c434737e6dc283..54b5a779029997 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir
@@ -30,7 +30,7 @@ body: |
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: DBG_VALUE
; GCN-NEXT: S_ENDPGM 0
bb.0:
@@ -46,12 +46,12 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
DBG_VALUE
bb.4:
DBG_VALUE
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -87,7 +87,7 @@ body: |
; GCN-NEXT: successors: %bb.5(0x80000000)
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
%0:sreg_64 = SI_IF undef %1:sreg_64, %bb.4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -100,12 +100,12 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -143,7 +143,7 @@ body: |
; GCN-NEXT: DBG_VALUE
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -158,13 +158,13 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
DBG_VALUE
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -204,7 +204,7 @@ body: |
; GCN-NEXT: KILL [[DEF]]
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -226,7 +226,7 @@ body: |
SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -268,7 +268,7 @@ body: |
; GCN-NEXT: [[COPY2:%[0-9]+]]:sgpr_32 = COPY [[S_BREV_B32_]]
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -283,14 +283,14 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%4:sgpr_32 = IMPLICIT_DEF
%5:sgpr_32 = S_BREV_B32 %4
KILL %4
%6:sgpr_32 = COPY %5
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -325,11 +325,11 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def $scc
; GCN-NEXT: [[S_BREV_B64_:%[0-9]+]]:sreg_64 = S_BREV_B64 $exec
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -344,11 +344,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%4:sreg_64 = S_BREV_B64 $exec
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -384,11 +384,11 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def $scc
; GCN-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY [[DEF]].sub2
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -404,11 +404,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%5:vgpr_32 = COPY %2.sub2
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -442,7 +442,7 @@ body: |
; GCN-NEXT: S_BRANCH %bb.5
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
@@ -462,11 +462,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
bb.5:
@@ -495,7 +495,7 @@ body: |
; GCN-NEXT: bb.1:
; GCN-NEXT: successors: %bb.1(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[V_CMP_LT_U32_e64_]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[V_CMP_LT_U32_e64_]], implicit-def $scc
; GCN-NEXT: S_BRANCH %bb.1
bb.0:
successors: %bb.1
@@ -507,7 +507,7 @@ body: |
bb.1:
successors: %bb.1
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.1
...
@@ -538,7 +538,7 @@ body: |
; GCN-NEXT: bb.2:
; GCN-NEXT: successors: %bb.3(0x40000000), %bb.6(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: [[S_AND_B64_1:%[0-9]+]]:sreg_64 = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def $scc
; GCN-NEXT: $exec = S_XOR_B64_term $exec, [[S_AND_B64_1]], implicit-def $scc
; GCN-NEXT: S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -557,10 +557,10 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.6:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.2
@@ -586,10 +586,10 @@ body: |
bb.5:
successors: %bb.6
- SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.6:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -622,12 +622,12 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
; GCN-NEXT: successors: %bb.5(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, %2, implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, %2, implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
@@ -652,10 +652,10 @@ body: |
S_BRANCH %bb.6
bb.3:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
@@ -698,7 +698,7 @@ body: |
; GCN-NEXT: bb.4:
; GCN-NEXT: successors: %bb.5(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
; GCN-NEXT: S_ENDPGM 0
@@ -715,11 +715,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
S_ENDPGM 0
@@ -765,7 +765,7 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.6:
; GCN-NEXT: successors: %bb.4(0x80000000)
@@ -784,7 +784,7 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.4:
@@ -792,7 +792,7 @@ body: |
bb.5:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.6:
S_BRANCH %bb.4
@@ -863,7 +863,7 @@ body: |
; GCN-NEXT: bb.7:
; GCN-NEXT: successors: %bb.8(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GCN-NEXT: S_BRANCH %bb.8
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.8:
@@ -896,14 +896,14 @@ body: |
; GCN-NEXT: bb.12:
; GCN-NEXT: successors: %bb.10(0x40000000), %bb.14(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: [[S_AND_B64_5:%[0-9]+]]:sreg_64 = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def $scc
; GCN-NEXT: $exec = S_XOR_B64_term $exec, [[S_AND_B64_5]], implicit-def $scc
; GCN-NEXT: S_CBRANCH_EXECZ %bb.14, implicit $exec
; GCN-NEXT: S_BRANCH %bb.10
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.14:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.14
@@ -945,19 +945,19 @@ body: |
bb.5:
successors: %bb.7
- SI_END_CF %11:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.7
bb.6:
successors: %bb.14
- SI_END_CF %5:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.14
bb.7:
successors: %bb.8
- SI_END_CF %8:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %8:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.8
bb.8:
@@ -992,12 +992,12 @@ body: |
bb.13:
successors: %bb.6
- SI_END_CF %15:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %15:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
bb.14:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir b/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir
index afa46f78778925..abc51f015fc12a 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir
@@ -64,12 +64,12 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY4]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY4]], implicit-def $scc
; GCN-NEXT: [[DEF:%[0-9]+]]:sreg_64 = IMPLICIT_DEF
; GCN-NEXT: dead [[SI_CALL:%[0-9]+]]:sreg_64 = SI_CALL [[DEF]], @func, csr_amdgpu
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GCN-NEXT: [[V_MOV_B32_e32_1:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 3, implicit $exec
; GCN-NEXT: [[V_MOV_B32_e32_2:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
; GCN-NEXT: $m0 = S_MOV_B32 -1
@@ -114,12 +114,12 @@ body: |
BUFFER_STORE_DWORD_ADDR64 %14, %8, %5, 0, 4, 0, 0, implicit $exec :: (store (s32), addrspace 1)
bb.3:
- $exec = S_OR_B64 $exec, %12, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %12, implicit-def $scc
%20:sreg_64 = IMPLICIT_DEF
%21:sreg_64 = SI_CALL %20, @func, csr_amdgpu
bb.4:
- $exec = S_OR_B64 $exec, %3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %3, implicit-def $scc
%17:vgpr_32 = V_MOV_B32_e32 3, implicit $exec
%18:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
$m0 = S_MOV_B32 -1
diff --git a/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir b/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir
index 5a2b055a110f8c..3e5453cb36728e 100644
--- a/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir
+++ b/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir
@@ -492,7 +492,7 @@ body: |
bb.3:
liveins: $vcc
- SI_END_CF %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit $vcc
...
diff --git a/llvm/test/CodeGen/AMDGPU/convergent.mir b/llvm/test/CodeGen/AMDGPU/convergent.mir
index 008d243e84309f..4a3a8342cacf38 100644
--- a/llvm/test/CodeGen/AMDGPU/convergent.mir
+++ b/llvm/test/CodeGen/AMDGPU/convergent.mir
@@ -45,7 +45,7 @@ body: |
%5:vreg_256 = COPY %3.sub1:vreg_256
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -81,7 +81,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -118,7 +118,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -153,7 +153,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -190,7 +190,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -233,7 +233,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -271,7 +271,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -309,7 +309,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -347,7 +347,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -385,7 +385,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -421,7 +421,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -457,7 +457,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -493,7 +493,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -534,7 +534,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -570,7 +570,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -606,7 +606,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -642,7 +642,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -678,6 +678,6 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll b/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll
index 4ffff7bd28ad4f..c1dd3679e859c1 100644
--- a/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll
+++ b/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll
@@ -149,8 +149,8 @@ define { i32, half } @call_split_type_used_outside_block_struct() #0 {
; GCN-NEXT: s_addc_u32 s17, s17, func_struct at rel32@hi+12
; GCN-NEXT: s_swappc_b64 s[30:31], s[16:17]
; GCN-NEXT: v_readlane_b32 s30, v40, 0
-; GCN-NEXT: v_readlane_b32 s31, v40, 1
; GCN-NEXT: v_mov_b32_e32 v1, v4
+; GCN-NEXT: v_readlane_b32 s31, v40, 1
; GCN-NEXT: s_mov_b32 s32, s33
; GCN-NEXT: v_readlane_b32 s4, v40, 2
; GCN-NEXT: s_or_saveexec_b64 s[6:7], -1
diff --git a/llvm/test/CodeGen/AMDGPU/div_i128.ll b/llvm/test/CodeGen/AMDGPU/div_i128.ll
index 5ff48c8c20c3e5..ff086c237c25a6 100644
--- a/llvm/test/CodeGen/AMDGPU/div_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/div_i128.ll
@@ -636,12 +636,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB0_4
; GFX9-O0-NEXT: .LBB0_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
@@ -666,8 +660,13 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -694,7 +693,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -704,7 +702,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -713,11 +710,9 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -852,9 +847,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB0_6
; GFX9-O0-NEXT: s_branch .LBB0_1
; GFX9-O0-NEXT: .LBB0_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
@@ -871,9 +863,12 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -947,6 +942,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -978,9 +974,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB0_6
; GFX9-O0-NEXT: .LBB0_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
@@ -989,14 +982,17 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -1106,6 +1102,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -1739,12 +1736,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_branch .LBB0_3
; GFX9-G-O0-NEXT: .LBB0_5: ; %udiv-do-while
; GFX9-G-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-G-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-G-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-G-O0-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
@@ -1769,12 +1760,16 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-G-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-G-O0-NEXT: s_mov_b64 s[8:9], 1
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(18)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v2
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v3
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v15, v5
; GFX9-G-O0-NEXT: v_mov_b32_e32 v14, v4
; GFX9-G-O0-NEXT: s_mov_b32 s11, 1
@@ -1810,10 +1805,8 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: v_lshrrev_b32_e64 v15, v2, v3
; GFX9-G-O0-NEXT: v_mov_b32_e32 v14, v0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v3, v1
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v27
; GFX9-G-O0-NEXT: v_mov_b32_e32 v2, v28
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v23, v29
; GFX9-G-O0-NEXT: v_mov_b32_e32 v24, v30
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v1
@@ -1838,7 +1831,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_sub_co_u32_e64 v13, s[12:13], v13, v4
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v12, s[12:13], v12, v9, s[12:13]
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v10, s[12:13], v10, v7, s[12:13]
@@ -1933,9 +1925,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_cbranch_execnz .LBB0_5
; GFX9-G-O0-NEXT: s_branch .LBB0_1
; GFX9-G-O0-NEXT: .LBB0_6: ; %udiv-preheader
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
@@ -1953,12 +1942,15 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b32 s4, 64
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v22, v17
; GFX9-G-O0-NEXT: v_mov_b32_e32 v23, v16
; GFX9-G-O0-NEXT: ; kill: def $vgpr20 killed $vgpr20 def $vgpr20_vgpr21 killed $exec
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v4
; GFX9-G-O0-NEXT: s_mov_b32 s5, 0xffffffc0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v4, s5
@@ -1970,7 +1962,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: v_cmp_lt_u32_e64 s[4:5], v18, v6
; GFX9-G-O0-NEXT: v_mov_b32_e32 v6, s6
; GFX9-G-O0-NEXT: v_cmp_eq_u32_e64 s[6:7], v18, v6
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-G-O0-NEXT: v_lshrrev_b64 v[6:7], v18, v[20:21]
; GFX9-G-O0-NEXT: v_lshrrev_b64 v[25:26], v18, v[22:23]
; GFX9-G-O0-NEXT: v_lshlrev_b64 v[23:24], v5, v[20:21]
@@ -2019,6 +2011,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], s[8:9]
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], s[8:9]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s8, 8
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s9, 9
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
@@ -2050,9 +2043,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_branch .LBB0_5
; GFX9-G-O0-NEXT: .LBB0_7: ; %udiv-bb1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
@@ -2061,15 +2051,18 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
; GFX9-G-O0-NEXT: s_mov_b32 s7, 1
; GFX9-G-O0-NEXT: s_mov_b32 s6, 0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v4, s7
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_add_co_u32_e64 v4, s[8:9], v2, v4
; GFX9-G-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: v_mov_b32_e32 v7, s6
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v5, s[8:9], v5, v7, s[8:9]
; GFX9-G-O0-NEXT: v_mov_b32_e32 v7, s6
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v7, s[8:9], v6, v7, s[8:9]
@@ -2164,6 +2157,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-G-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-G-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(17)
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
@@ -2738,12 +2732,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB1_4
; GFX9-O0-NEXT: .LBB1_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
@@ -2768,8 +2756,13 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -2796,7 +2789,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -2806,7 +2798,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -2815,11 +2806,9 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -2954,9 +2943,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB1_6
; GFX9-O0-NEXT: s_branch .LBB1_1
; GFX9-O0-NEXT: .LBB1_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
@@ -2973,9 +2959,12 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -3049,6 +3038,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -3080,9 +3070,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB1_6
; GFX9-O0-NEXT: .LBB1_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
@@ -3091,14 +3078,17 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -3208,6 +3198,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -3755,12 +3746,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_branch .LBB1_3
; GFX9-G-O0-NEXT: .LBB1_5: ; %udiv-do-while
; GFX9-G-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-G-O0-NEXT: v_readlane_b32 s6, v32, 8
-; GFX9-G-O0-NEXT: v_readlane_b32 s7, v32, 9
; GFX9-G-O0-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
@@ -3785,12 +3770,16 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: v_readlane_b32 s6, v32, 8
+; GFX9-G-O0-NEXT: v_readlane_b32 s7, v32, 9
; GFX9-G-O0-NEXT: s_mov_b64 s[8:9], 1
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(18)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v2
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v3
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v5
; GFX9-G-O0-NEXT: v_mov_b32_e32 v20, v4
; GFX9-G-O0-NEXT: s_mov_b32 s11, 1
@@ -3826,10 +3815,8 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: v_lshrrev_b32_e64 v13, v2, v3
; GFX9-G-O0-NEXT: v_mov_b32_e32 v12, v0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v3, v1
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v28
; GFX9-G-O0-NEXT: v_mov_b32_e32 v2, v29
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v20, v30
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v31
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v1
@@ -3854,7 +3841,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_sub_co_u32_e64 v11, s[12:13], v11, v4
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v10, s[12:13], v10, v9, s[12:13]
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v8, s[12:13], v8, v7, s[12:13]
@@ -3957,9 +3943,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_cbranch_execnz .LBB1_5
; GFX9-G-O0-NEXT: s_branch .LBB1_1
; GFX9-G-O0-NEXT: .LBB1_6: ; %udiv-preheader
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
@@ -3977,11 +3960,14 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b32 s4, 64
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v15, v5
; GFX9-G-O0-NEXT: v_mov_b32_e32 v14, v4
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v7
; GFX9-G-O0-NEXT: v_mov_b32_e32 v20, v6
; GFX9-G-O0-NEXT: s_mov_b32 s5, 0xffffffc0
@@ -4048,6 +4034,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], s[8:9]
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], s[8:9]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s8, 8
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s9, 9
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
@@ -4079,9 +4066,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_branch .LBB1_5
; GFX9-G-O0-NEXT: .LBB1_7: ; %udiv-bb1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:52 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
@@ -4090,15 +4074,18 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
; GFX9-G-O0-NEXT: s_mov_b32 s7, 1
; GFX9-G-O0-NEXT: s_mov_b32 s6, 0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v4, s7
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_add_co_u32_e64 v4, s[8:9], v1, v4
; GFX9-G-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: v_mov_b32_e32 v5, s6
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v5, s[8:9], v3, v5, s[8:9]
; GFX9-G-O0-NEXT: v_mov_b32_e32 v3, s6
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v7, s[8:9], v2, v3, s[8:9]
@@ -4193,6 +4180,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-G-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-G-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(17)
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s6, 6
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s7, 7
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
diff --git a/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir b/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir
index 41127424dd8926..477e7233d9e013 100644
--- a/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir
+++ b/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir
@@ -48,7 +48,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%8:vgpr_32 = COPY %7
%9:sreg_64_xexec = IMPLICIT_DEF
%10:vgpr_32 = V_SET_INACTIVE_B32 0, %8, 0, 0, killed %9, implicit $exec
@@ -114,7 +114,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%11:vgpr_32 = V_ADD_U32_e64 %9, 1, 0, implicit $exec
$vgpr0 = COPY %11
SI_RETURN implicit $vgpr0
@@ -177,7 +177,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%11:vgpr_32 = COPY %9
%12:vgpr_32 = V_ADD_U32_e64 %11, 1, 0, implicit $exec
$vgpr0 = COPY %12
diff --git a/llvm/test/CodeGen/AMDGPU/dpp_combine.mir b/llvm/test/CodeGen/AMDGPU/dpp_combine.mir
index fe9d6864c5d52d..0d061c7d42dce1 100644
--- a/llvm/test/CodeGen/AMDGPU/dpp_combine.mir
+++ b/llvm/test/CodeGen/AMDGPU/dpp_combine.mir
@@ -435,7 +435,7 @@ body: |
successors: %bb.2
bb.2:
- SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
...
# GCN-LABEL: name: old_in_diff_bb
diff --git a/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir b/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir
index 321414a81bdc8a..5a62c9ce1cbdcc 100644
--- a/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir
+++ b/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir
@@ -393,7 +393,7 @@ body: |
successors: %bb.2
bb.2:
- SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
...
# GCN-LABEL: name: old_in_diff_bb
diff --git a/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir b/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir
index 3433534698dd4e..49f59cb7ec41df 100644
--- a/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir
+++ b/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir
@@ -176,7 +176,7 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: liveins: $sgpr6_sgpr7
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
; GCN-NEXT: SI_RETURN
bb.0:
liveins: $sgpr4, $sgpr10_sgpr11
@@ -208,7 +208,7 @@ body: |
S_BRANCH %bb.3
bb.5:
liveins: $sgpr6_sgpr7
- $exec = S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll b/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll
index aabed8e2814c0d..5c8ac8dcc14d4f 100644
--- a/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll
+++ b/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll
@@ -2963,12 +2963,6 @@ define float @float32_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB20_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[46:47], -1
-; GCN-O0-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[46:47]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v63, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v63, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
@@ -3002,7 +2996,12 @@ define float @float32_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[46:47], -1
+; GCN-O0-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[46:47]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v63, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v63, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v32
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v32
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3375,12 +3374,6 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB21_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
-; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v34, 35
-; GCN-O0-NEXT: v_readlane_b32 s5, v34, 36
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -3414,7 +3407,12 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
+; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v34, 35
+; GCN-O0-NEXT: v_readlane_b32 s5, v34, 36
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v32
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v32
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3547,12 +3545,6 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB21_4: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
-; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v34, 39
-; GCN-O0-NEXT: v_readlane_b32 s5, v34, 40
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
@@ -3586,7 +3578,12 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
+; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v34, 39
+; GCN-O0-NEXT: v_readlane_b32 s5, v34, 40
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v32
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v32
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3687,17 +3684,16 @@ define i32 @extract_dyn_i32_3(<3 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB22_1: ; =>This Inner Loop Header: Depth=1
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 2
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 3
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v3
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v3
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3799,17 +3795,16 @@ define float @extract_dyn_float_3(<3 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB24_1: ; =>This Inner Loop Header: Depth=1
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 2
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 3
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v3
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v3
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3927,19 +3922,18 @@ define i32 @extract_dyn_i32_5(<5 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB26_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v5
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v5
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4061,19 +4055,18 @@ define float @extract_dyn_float_5(<5 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB28_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v5
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v5
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4209,12 +4202,6 @@ define i32 @extract_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB30_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4222,7 +4209,12 @@ define i32 @extract_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4356,12 +4348,6 @@ define float @extract_dyn_float_6(<6 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB32_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4369,7 +4355,12 @@ define float @extract_dyn_float_6(<6 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4519,12 +4510,6 @@ define i32 @extract_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB34_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4533,7 +4518,12 @@ define i32 @extract_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4675,12 +4665,6 @@ define float @extract_dyn_float_7(<7 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB36_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4689,7 +4673,12 @@ define float @extract_dyn_float_7(<7 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
diff --git a/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir b/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir
index 6584abbb5716c1..f1507dd420740f 100644
--- a/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir
+++ b/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir
@@ -571,7 +571,7 @@ body: |
; CHECK-NEXT: SI_WATERFALL_LOOP %bb.1, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec = S_MOV_B64 [[S_MOV_B64_]]
+ ; CHECK-NEXT: $exec = bb-prolog S_MOV_B64 [[S_MOV_B64_]]
; CHECK-NEXT: SI_RETURN
bb.0:
liveins: $vgpr0_vgpr1
@@ -595,7 +595,7 @@ body: |
SI_WATERFALL_LOOP %bb.1, implicit $exec
bb.2:
- $exec = S_MOV_B64 %3
+ $exec = bb-prolog S_MOV_B64 %3
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir b/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir
index 9297b44695c686..aa49187d4916e1 100644
--- a/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir
+++ b/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir
@@ -29,7 +29,7 @@ body: |
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.5(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr4_sgpr5 = S_OR_SAVEEXEC_B64 undef renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: renamable $sgpr4_sgpr5 = bb-prolog S_OR_SAVEEXEC_B64 undef renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: $exec = S_XOR_B64_term $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; CHECK-NEXT: {{ $}}
@@ -67,7 +67,7 @@ body: |
S_CBRANCH_EXECZ %bb.3, implicit $exec
bb.1:
- %1:sreg_64 = S_OR_SAVEEXEC_B64 undef %2:sreg_64, implicit-def $exec, implicit-def $scc, implicit $exec
+ %1:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 undef %2:sreg_64, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %1, implicit-def $scc
S_CBRANCH_EXECZ %bb.5, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir b/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir
index 94c22b1aa8664f..f73316636e7b24 100644
--- a/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir
+++ b/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir
@@ -94,7 +94,7 @@ body: |
; CHECK-NEXT: successors: %bb.5(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $sgpr4_sgpr5, $sgpr64_sgpr65:0x000000000000000F, $sgpr68_sgpr69_sgpr70_sgpr71_sgpr72_sgpr73_sgpr74_sgpr75_sgpr76_sgpr77_sgpr78_sgpr79_sgpr80_sgpr81_sgpr82_sgpr83_sgpr84_sgpr85_sgpr86_sgpr87_sgpr88_sgpr89_sgpr90_sgpr91_sgpr92_sgpr93_sgpr94_sgpr95_sgpr96_sgpr97_sgpr98_sgpr99:0x0000000000000003
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr6_sgpr7 = S_OR_SAVEEXEC_B64 renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: renamable $sgpr6_sgpr7 = bb-prolog S_OR_SAVEEXEC_B64 renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: renamable $sgpr36 = COPY renamable $sgpr68
; CHECK-NEXT: renamable $sgpr37 = COPY renamable $sgpr68
; CHECK-NEXT: renamable $sgpr38 = COPY renamable $sgpr68
@@ -126,7 +126,7 @@ body: |
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: liveins: $sgpr6_sgpr7, $sgpr64_sgpr65:0x0000000000000003
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: dead renamable $sgpr4 = S_LSHL_B32 killed renamable $sgpr64, 1, implicit-def dead $scc
; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.0, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.0, align 4, addrspace 5)
; CHECK-NEXT: dead [[COPY2:%[0-9]+]]:vreg_1024 = COPY killed renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
@@ -191,7 +191,7 @@ body: |
%12:vreg_1024 = V_INDIRECT_REG_WRITE_GPR_IDX_B32_V32 %12, 0, %13, 3, implicit-def $m0, implicit $m0, implicit $exec
bb.3:
- %14:sreg_64 = S_OR_SAVEEXEC_B64 %9, implicit-def $exec, implicit-def $scc, implicit $exec
+ %14:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %9, implicit-def $exec, implicit-def $scc, implicit $exec
undef %15.sub0:sgpr_1024 = COPY %7.sub0
%15.sub1:sgpr_1024 = COPY %7.sub0
%15.sub2:sgpr_1024 = COPY %7.sub0
@@ -220,7 +220,7 @@ body: |
S_BRANCH %bb.1
bb.4:
- $exec = S_OR_B64 $exec, %11, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %11, implicit-def $scc
%17:sreg_32 = S_LSHL_B32 %6.sub0, 1, implicit-def dead $scc
%16:vreg_1024 = COPY %7
diff --git a/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir b/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir
index caa4c87eacc62b..030dc20accdf88 100644
--- a/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir
+++ b/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir
@@ -33,7 +33,7 @@ body: |
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.3:
- $exec_lo = S_OR_B32 $exec_lo, undef $sgpr4, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, undef $sgpr4, implicit-def $scc
$vgpr0 = COPY %0
S_SETPC_B64_return undef $sgpr30_sgpr31, implicit $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll b/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
index 127f4996b7676b..e339bf976b5e79 100644
--- a/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
+++ b/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
@@ -1497,13 +1497,7 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: ; implicit-def: $vgpr0
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB5_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 6
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 7
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[20:23], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[20:23], 0 offset:12 ; 4-byte Folded Reload
@@ -1514,14 +1508,26 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: buffer_load_dword v7, off, s[20:23], 0 offset:32 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v8, off, s[20:23], 0 offset:36 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v9, off, s[20:23], 0 offset:40 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(6)
; NOOPT-NEXT: buffer_load_dword v10, off, s[20:23], 0 offset:44 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(5)
; NOOPT-NEXT: buffer_load_dword v11, off, s[20:23], 0 offset:48 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(4)
; NOOPT-NEXT: buffer_load_dword v12, off, s[20:23], 0 offset:52 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(3)
; NOOPT-NEXT: buffer_load_dword v13, off, s[20:23], 0 offset:56 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(2)
; NOOPT-NEXT: buffer_load_dword v14, off, s[20:23], 0 offset:60 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:68 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 6
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 7
; NOOPT-NEXT: v_readfirstlane_b32 s2, v16
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v16
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -1546,6 +1552,7 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: v_readlane_b32 s1, v31, 5
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
+; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:72 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
@@ -1554,8 +1561,6 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: v_readlane_b32 s1, v31, 1
; NOOPT-NEXT: v_readlane_b32 s2, v31, 2
; NOOPT-NEXT: v_readlane_b32 s3, v31, 3
-; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:72 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v0, off, s[0:3], 0
; NOOPT-NEXT: s_endpgm
;
@@ -3888,13 +3893,6 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: buffer_store_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB14_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[20:23], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[20:23], 0 offset:12 ; 4-byte Folded Reload
@@ -3920,7 +3918,12 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:68 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:136 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -3976,17 +3979,6 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: v_readlane_b32 s1, v31, 8
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
-; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
-; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
-; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
-; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
-; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:148 ; 4-byte Folded Reload
@@ -4003,22 +3995,29 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: buffer_load_dword v28, off, s[20:23], 0 offset:192 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v29, off, s[20:23], 0 offset:196 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v30, off, s[20:23], 0 offset:200 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
+; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
+; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
+; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
+; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
+; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: v_mov_b32_e32 v4, v18
; NOOPT-NEXT: v_mov_b32_e32 v5, v17
; NOOPT-NEXT: v_mov_b32_e32 v6, v16
; NOOPT-NEXT: v_mov_b32_e32 v0, v15
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v1, v22
; NOOPT-NEXT: v_mov_b32_e32 v2, v21
; NOOPT-NEXT: v_mov_b32_e32 v3, v20
; NOOPT-NEXT: v_mov_b32_e32 v7, v19
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v12, v26
; NOOPT-NEXT: v_mov_b32_e32 v13, v25
; NOOPT-NEXT: v_mov_b32_e32 v14, v24
; NOOPT-NEXT: v_mov_b32_e32 v8, v23
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v9, v30
; NOOPT-NEXT: v_mov_b32_e32 v10, v29
; NOOPT-NEXT: v_mov_b32_e32 v11, v28
@@ -4377,13 +4376,6 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: buffer_store_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB15_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[20:23], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[20:23], 0 offset:12 ; 4-byte Folded Reload
@@ -4409,7 +4401,12 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:68 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:136 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -4465,17 +4462,6 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: v_readlane_b32 s1, v31, 8
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
-; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
-; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
-; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
-; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
-; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:148 ; 4-byte Folded Reload
@@ -4492,22 +4478,29 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: buffer_load_dword v28, off, s[20:23], 0 offset:192 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v29, off, s[20:23], 0 offset:196 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v30, off, s[20:23], 0 offset:200 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
+; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
+; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
+; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
+; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
+; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: v_mov_b32_e32 v4, v18
; NOOPT-NEXT: v_mov_b32_e32 v5, v17
; NOOPT-NEXT: v_mov_b32_e32 v6, v16
; NOOPT-NEXT: v_mov_b32_e32 v0, v15
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v1, v22
; NOOPT-NEXT: v_mov_b32_e32 v2, v21
; NOOPT-NEXT: v_mov_b32_e32 v3, v20
; NOOPT-NEXT: v_mov_b32_e32 v7, v19
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v12, v26
; NOOPT-NEXT: v_mov_b32_e32 v13, v25
; NOOPT-NEXT: v_mov_b32_e32 v14, v24
; NOOPT-NEXT: v_mov_b32_e32 v8, v23
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v9, v30
; NOOPT-NEXT: v_mov_b32_e32 v10, v29
; NOOPT-NEXT: v_mov_b32_e32 v11, v28
@@ -4932,13 +4925,7 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: ; implicit-def: $vgpr0
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB16_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v17, 23
-; NOOPT-NEXT: v_readlane_b32 s1, v17, 24
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:12 ; 4-byte Folded Reload
@@ -4949,14 +4936,26 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: buffer_load_dword v7, off, s[36:39], 0 offset:32 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v8, off, s[36:39], 0 offset:36 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v9, off, s[36:39], 0 offset:40 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(6)
; NOOPT-NEXT: buffer_load_dword v10, off, s[36:39], 0 offset:44 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(5)
; NOOPT-NEXT: buffer_load_dword v11, off, s[36:39], 0 offset:48 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(4)
; NOOPT-NEXT: buffer_load_dword v12, off, s[36:39], 0 offset:52 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(3)
; NOOPT-NEXT: buffer_load_dword v13, off, s[36:39], 0 offset:56 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(2)
; NOOPT-NEXT: buffer_load_dword v14, off, s[36:39], 0 offset:60 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[36:39], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[36:39], 0 offset:72 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v17, 23
+; NOOPT-NEXT: v_readlane_b32 s1, v17, 24
; NOOPT-NEXT: v_readfirstlane_b32 s2, v16
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v16
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5047,13 +5046,7 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: ; implicit-def: $vgpr0
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB16_4: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v17, 28
-; NOOPT-NEXT: v_readlane_b32 s1, v17, 29
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:84 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:88 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:92 ; 4-byte Folded Reload
@@ -5064,14 +5057,26 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: buffer_load_dword v7, off, s[36:39], 0 offset:112 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v8, off, s[36:39], 0 offset:116 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v9, off, s[36:39], 0 offset:120 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(6)
; NOOPT-NEXT: buffer_load_dword v10, off, s[36:39], 0 offset:124 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(5)
; NOOPT-NEXT: buffer_load_dword v11, off, s[36:39], 0 offset:128 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(4)
; NOOPT-NEXT: buffer_load_dword v12, off, s[36:39], 0 offset:132 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(3)
; NOOPT-NEXT: buffer_load_dword v13, off, s[36:39], 0 offset:136 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(2)
; NOOPT-NEXT: buffer_load_dword v14, off, s[36:39], 0 offset:140 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[36:39], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[36:39], 0 offset:68 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v17, 28
+; NOOPT-NEXT: v_readlane_b32 s1, v17, 29
; NOOPT-NEXT: v_readfirstlane_b32 s2, v16
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v16
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5096,6 +5101,9 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: v_readlane_b32 s1, v17, 27
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.6:
+; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:76 ; 4-byte Folded Reload
+; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:148 ; 4-byte Folded Reload
+; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:80 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
@@ -5105,10 +5113,6 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: v_readlane_b32 s5, v17, 1
; NOOPT-NEXT: v_readlane_b32 s6, v17, 2
; NOOPT-NEXT: v_readlane_b32 s7, v17, 3
-; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:76 ; 4-byte Folded Reload
-; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:148 ; 4-byte Folded Reload
-; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:80 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v2, off, s[4:7], 0
; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v1, off, s[4:7], 0
@@ -5668,13 +5672,6 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_store_dword v15, off, s[28:31], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB17_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v32, 11
-; NOOPT-NEXT: v_readlane_b32 s1, v32, 12
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[28:31], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[28:31], 0 offset:12 ; 4-byte Folded Reload
@@ -5700,7 +5697,12 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[28:31], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[28:31], 0 offset:72 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[28:31], 0 offset:80 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v32, 11
+; NOOPT-NEXT: v_readlane_b32 s1, v32, 12
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5756,9 +5758,6 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: v_readlane_b32 s1, v32, 10
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:88 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[28:31], 0 offset:92 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[28:31], 0 offset:96 ; 4-byte Folded Reload
@@ -5775,10 +5774,13 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v13, off, s[28:31], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v14, off, s[28:31], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v15, off, s[28:31], 0 offset:148 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: v_mov_b32_e32 v16, 63
; NOOPT-NEXT: buffer_store_dword v16, off, s[28:31], 0 offset:216 ; 4-byte Folded Spill
; NOOPT-NEXT: s_mov_b64 s[0:1], exec
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
+; NOOPT-NEXT: s_waitcnt vmcnt(1)
; NOOPT-NEXT: v_writelane_b32 v32, s0, 13
; NOOPT-NEXT: v_writelane_b32 v32, s1, 14
; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
@@ -5787,32 +5789,21 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_store_dword v0, off, s[28:31], 0 offset:152 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v1, off, s[28:31], 0 offset:156 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v2, off, s[28:31], 0 offset:160 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v3, off, s[28:31], 0 offset:164 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v4, off, s[28:31], 0 offset:168 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v5, off, s[28:31], 0 offset:172 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v6, off, s[28:31], 0 offset:176 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v7, off, s[28:31], 0 offset:180 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v8, off, s[28:31], 0 offset:184 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v9, off, s[28:31], 0 offset:188 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v10, off, s[28:31], 0 offset:192 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v11, off, s[28:31], 0 offset:196 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v12, off, s[28:31], 0 offset:200 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v13, off, s[28:31], 0 offset:204 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v14, off, s[28:31], 0 offset:208 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v15, off, s[28:31], 0 offset:212 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB17_4: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v32, 15
-; NOOPT-NEXT: v_readlane_b32 s1, v32, 16
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:152 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[28:31], 0 offset:156 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[28:31], 0 offset:160 ; 4-byte Folded Reload
@@ -5838,7 +5829,12 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[28:31], 0 offset:212 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[28:31], 0 offset:216 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[28:31], 0 offset:76 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v32, 15
+; NOOPT-NEXT: v_readlane_b32 s1, v32, 16
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5894,19 +5890,6 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: v_readlane_b32 s1, v32, 14
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.6:
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v32, 6
-; NOOPT-NEXT: v_readlane_b32 s4, v32, 2
-; NOOPT-NEXT: v_readlane_b32 s5, v32, 3
-; NOOPT-NEXT: v_readlane_b32 s6, v32, 4
-; NOOPT-NEXT: v_readlane_b32 s7, v32, 5
-; NOOPT-NEXT: v_readlane_b32 s2, v32, 7
-; NOOPT-NEXT: v_readlane_b32 s3, v32, 8
-; NOOPT-NEXT: v_readlane_b32 s8, v32, 0
-; NOOPT-NEXT: v_readlane_b32 s9, v32, 1
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:84 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[28:31], 0 offset:220 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[28:31], 0 offset:224 ; 4-byte Folded Reload
@@ -5924,22 +5907,31 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v29, off, s[28:31], 0 offset:272 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v30, off, s[28:31], 0 offset:276 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v31, off, s[28:31], 0 offset:280 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v32, 6
+; NOOPT-NEXT: v_readlane_b32 s4, v32, 2
+; NOOPT-NEXT: v_readlane_b32 s5, v32, 3
+; NOOPT-NEXT: v_readlane_b32 s6, v32, 4
+; NOOPT-NEXT: v_readlane_b32 s7, v32, 5
+; NOOPT-NEXT: v_readlane_b32 s2, v32, 7
+; NOOPT-NEXT: v_readlane_b32 s3, v32, 8
+; NOOPT-NEXT: v_readlane_b32 s8, v32, 0
+; NOOPT-NEXT: v_readlane_b32 s9, v32, 1
; NOOPT-NEXT: v_mov_b32_e32 v5, v19
; NOOPT-NEXT: v_mov_b32_e32 v6, v18
; NOOPT-NEXT: v_mov_b32_e32 v7, v17
; NOOPT-NEXT: v_mov_b32_e32 v1, v16
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v2, v23
; NOOPT-NEXT: v_mov_b32_e32 v3, v22
; NOOPT-NEXT: v_mov_b32_e32 v4, v21
; NOOPT-NEXT: v_mov_b32_e32 v8, v20
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v13, v27
; NOOPT-NEXT: v_mov_b32_e32 v14, v26
; NOOPT-NEXT: v_mov_b32_e32 v15, v25
; NOOPT-NEXT: v_mov_b32_e32 v9, v24
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v10, v31
; NOOPT-NEXT: v_mov_b32_e32 v11, v30
; NOOPT-NEXT: v_mov_b32_e32 v12, v29
@@ -8786,17 +8778,15 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: .LBB26_1: ; %bb2
; NOOPT-NEXT: ; =>This Loop Header: Depth=1
; NOOPT-NEXT: ; Child Loop BB26_3 Depth 2
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
-; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_readlane_b32 s3, v18, 0
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 s[0:1], -1
; NOOPT-NEXT: ; implicit-def: $sgpr2
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_cmp_lt_i32_e64 s[4:5], v0, s3
; NOOPT-NEXT: s_xor_b64 s[4:5], s[4:5], s[0:1]
; NOOPT-NEXT: s_and_b64 vcc, exec, s[4:5]
@@ -8874,13 +8864,6 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB26_3: ; Parent Loop BB26_1 Depth=1
; NOOPT-NEXT: ; => This Inner Loop Header: Depth=2
-; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v18, 6
-; NOOPT-NEXT: v_readlane_b32 s1, v18, 7
; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:12 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[24:27], 0 offset:16 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[24:27], 0 offset:20 ; 4-byte Folded Reload
@@ -8906,7 +8889,12 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: buffer_load_dword v15, off, s[24:27], 0 offset:72 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[24:27], 0 offset:76 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[24:27], 0 offset:80 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
+; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v18, 6
+; NOOPT-NEXT: v_readlane_b32 s1, v18, 7
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -8962,9 +8950,6 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: v_readlane_b32 s1, v18, 5
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.5: ; in Loop: Header=BB26_1 Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
-; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:84 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[24:27], 0 offset:88 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[24:27], 0 offset:92 ; 4-byte Folded Reload
@@ -8981,9 +8966,13 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: buffer_load_dword v13, off, s[24:27], 0 offset:136 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v14, off, s[24:27], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v15, off, s[24:27], 0 offset:144 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
+; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: s_mov_b64 s[0:1], 0
; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v0, off, s[24:27], 0 offset:8 ; 4-byte Folded Spill
+; NOOPT-NEXT: s_waitcnt vmcnt(1)
; NOOPT-NEXT: v_writelane_b32 v18, s0, 2
; NOOPT-NEXT: v_writelane_b32 v18, s1, 3
; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -8991,6 +8980,8 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: .LBB26_6: ; %Flow
; NOOPT-NEXT: ; in Loop: Header=BB26_1 Depth=1
+; NOOPT-NEXT: s_waitcnt expcnt(1)
+; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
; NOOPT-NEXT: s_waitcnt expcnt(0)
; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
@@ -8998,11 +8989,9 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_readlane_b32 s0, v18, 2
; NOOPT-NEXT: v_readlane_b32 s1, v18, 3
-; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 s[2:3], -1
; NOOPT-NEXT: s_xor_b64 s[0:1], s[0:1], s[2:3]
; NOOPT-NEXT: s_and_b64 vcc, exec, s[0:1]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v0, off, s[24:27], 0 offset:4 ; 4-byte Folded Spill
; NOOPT-NEXT: s_cbranch_vccnz .LBB26_1
; NOOPT-NEXT: ; %bb.7: ; %bb8
@@ -9363,13 +9352,6 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: buffer_store_dword v15, off, s[16:19], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB27_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v33, 11
-; NOOPT-NEXT: v_readlane_b32 s1, v33, 12
; NOOPT-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[16:19], 0 offset:12 ; 4-byte Folded Reload
@@ -9395,7 +9377,12 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: buffer_load_dword v15, off, s[16:19], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[16:19], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[16:19], 0 offset:132 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
+; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v33, 11
+; NOOPT-NEXT: v_readlane_b32 s1, v33, 12
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -9451,16 +9438,6 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: v_readlane_b32 s1, v33, 10
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
-; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v33, 5
-; NOOPT-NEXT: v_readlane_b32 s1, v33, 6
-; NOOPT-NEXT: v_readlane_b32 s2, v33, 7
-; NOOPT-NEXT: v_readlane_b32 s3, v33, 8
-; NOOPT-NEXT: v_readlane_b32 s8, v33, 3
-; NOOPT-NEXT: v_readlane_b32 s9, v33, 4
; NOOPT-NEXT: buffer_load_dword v4, off, s[16:19], 0 offset:136 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v5, off, s[16:19], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[16:19], 0 offset:148 ; 4-byte Folded Reload
@@ -9479,22 +9456,28 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: buffer_load_dword v30, off, s[16:19], 0 offset:200 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v31, off, s[16:19], 0 offset:204 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v32, off, s[16:19], 0 offset:208 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
+; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v33, 5
+; NOOPT-NEXT: v_readlane_b32 s1, v33, 6
+; NOOPT-NEXT: v_readlane_b32 s2, v33, 7
+; NOOPT-NEXT: v_readlane_b32 s3, v33, 8
+; NOOPT-NEXT: v_readlane_b32 s8, v33, 3
+; NOOPT-NEXT: v_readlane_b32 s9, v33, 4
; NOOPT-NEXT: v_mov_b32_e32 v6, v20
; NOOPT-NEXT: v_mov_b32_e32 v7, v19
; NOOPT-NEXT: v_mov_b32_e32 v8, v18
; NOOPT-NEXT: v_mov_b32_e32 v0, v17
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v1, v24
; NOOPT-NEXT: v_mov_b32_e32 v2, v23
; NOOPT-NEXT: v_mov_b32_e32 v3, v22
; NOOPT-NEXT: v_mov_b32_e32 v9, v21
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v14, v28
; NOOPT-NEXT: v_mov_b32_e32 v15, v27
; NOOPT-NEXT: v_mov_b32_e32 v16, v26
; NOOPT-NEXT: v_mov_b32_e32 v10, v25
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v11, v32
; NOOPT-NEXT: v_mov_b32_e32 v12, v31
; NOOPT-NEXT: v_mov_b32_e32 v13, v30
diff --git a/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir b/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir
index 3081a2c1d1fe1a..8773de86bf44bb 100644
--- a/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir
+++ b/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir
@@ -167,7 +167,7 @@ body: |
...
---
# GCN-LABEL: name: and_execz_mov_exec_vccz
-# GCN: $exec = S_MOV_B64 -1
+# GCN: $exec = bb-prolog S_MOV_B64 -1
# GCN-NEXT: S_ENDPGM 0
name: and_execz_mov_exec_vccz
body: |
@@ -178,14 +178,14 @@ body: |
S_NOP 0
bb.2:
- $exec = S_MOV_B64 -1
+ $exec = bb-prolog S_MOV_B64 -1
$vcc = S_AND_B64 $exec, $exec, implicit-def dead $scc
S_CBRANCH_VCCZ %bb.1, implicit killed $vcc
S_ENDPGM 0
...
---
# GCN-LABEL: name: and_execz_mov_exec_vccnz
-# GCN: $exec = S_MOV_B64 -1
+# GCN: $exec = bb-prolog S_MOV_B64 -1
# GCN-NEXT: S_BRANCH %bb.1{{$}}
name: and_execz_mov_exec_vccnz
body: |
@@ -196,7 +196,7 @@ body: |
S_NOP 0
bb.2:
- $exec = S_MOV_B64 -1
+ $exec = bb-prolog S_MOV_B64 -1
$vcc = S_AND_B64 $exec, $exec, implicit-def dead $scc
S_CBRANCH_VCCNZ %bb.1, implicit killed $vcc
S_ENDPGM 0
@@ -626,7 +626,7 @@ body: |
S_NOP 0
bb.2:
- V_CMPX_EQ_U32_e32 0, killed $vgpr0, implicit-def $vcc, implicit-def $exec, implicit $exec
+ bb-prolog V_CMPX_EQ_U32_e32 0, killed $vgpr0, implicit-def $vcc, implicit-def $exec, implicit $exec
$vcc = S_AND_B64 $exec, $vcc, implicit-def dead $scc
S_CBRANCH_VCCZ %bb.1, implicit killed $vcc
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll b/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll
index fadb15dd2fb799..d82cdd26b6643a 100644
--- a/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll
+++ b/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll
@@ -64,7 +64,7 @@ define amdgpu_kernel void @widget(ptr addrspace(1) %arg, i1 %arg1) #0 {
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $vgpr0, $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9, $vgpr0_vgpr1:0x000000000000000C, $vgpr2_vgpr3, $sgpr20_sgpr21_sgpr22_sgpr23
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr8_sgpr9, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr8_sgpr9, implicit-def $scc
; CHECK-NEXT: renamable $sgpr8_sgpr9 = S_AND_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: renamable $sgpr4_sgpr5 = S_OR_B64 killed renamable $sgpr8_sgpr9, killed renamable $sgpr4_sgpr5, implicit-def $scc
; CHECK-NEXT: $exec = S_ANDN2_B64 $exec, renamable $sgpr4_sgpr5, implicit-def $scc
@@ -74,7 +74,7 @@ define amdgpu_kernel void @widget(ptr addrspace(1) %arg, i1 %arg1) #0 {
; CHECK-NEXT: successors: %bb.4(0x80000000)
; CHECK-NEXT: liveins: $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5, $vgpr0_vgpr1:0x000000000000000C, $vgpr2_vgpr3, $sgpr20_sgpr21_sgpr22_sgpr23
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; CHECK-NEXT: renamable $vgpr4 = GLOBAL_LOAD_DWORD renamable $vgpr2_vgpr3, 0, 0, implicit $exec :: (load (s32) from `ptr addrspace(1) null`, addrspace 1)
; CHECK-NEXT: renamable $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr20_sgpr21_sgpr22_sgpr23, 0, 0, 0, 0, implicit $exec :: (load (s32) from `ptr addrspace(5) null`, addrspace 5)
; CHECK-NEXT: S_WAITCNT .Vmcnt_1
diff --git a/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll b/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll
index e756e485a75c80..9a1121f4fac4c0 100644
--- a/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll
+++ b/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll
@@ -8205,12 +8205,6 @@ define amdgpu_ps <32 x float> @float32_inselt_vec(<32 x float> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_store_dword v31, off, s[8:11], 0 offset:128 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr0_sgpr1
; GCN-O0-NEXT: .LBB22_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[4:5], -1
-; GCN-O0-NEXT: buffer_load_dword v63, off, s[8:11], 0 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[4:5]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s0, v63, 2
-; GCN-O0-NEXT: v_readlane_b32 s1, v63, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[8:11], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[8:11], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[8:11], 0 offset:12 ; 4-byte Folded Reload
@@ -8245,7 +8239,12 @@ define amdgpu_ps <32 x float> @float32_inselt_vec(<32 x float> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v31, off, s[8:11], 0 offset:128 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[8:11], 0 offset:132 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v33, off, s[8:11], 0 offset:264 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[4:5], -1
+; GCN-O0-NEXT: buffer_load_dword v63, off, s[8:11], 0 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[4:5]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s0, v63, 2
+; GCN-O0-NEXT: v_readlane_b32 s1, v63, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s2, v33
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v33
; GCN-O0-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -8534,12 +8533,6 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB23_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v32, 4
-; GCN-O0-NEXT: v_readlane_b32 s5, v32, 5
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -8558,7 +8551,12 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v32, 4
+; GCN-O0-NEXT: v_readlane_b32 s5, v32, 5
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v17
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v17
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -8613,12 +8611,6 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: v_readlane_b32 s5, v32, 3
; GCN-O0-NEXT: s_mov_b64 exec, s[4:5]
; GCN-O0-NEXT: ; %bb.3:
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v32, 0
-; GCN-O0-NEXT: v_readlane_b32 s5, v32, 1
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
@@ -8636,8 +8628,13 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b32 s6, 1
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v32, 0
+; GCN-O0-NEXT: v_readlane_b32 s5, v32, 1
+; GCN-O0-NEXT: s_mov_b32 s6, 1
; GCN-O0-NEXT: v_add_u32_e64 v16, s[6:7], v16, s6
; GCN-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
; GCN-O0-NEXT: s_mov_b32 s4, s5
@@ -8667,12 +8664,6 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB23_4: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v32, 8
-; GCN-O0-NEXT: v_readlane_b32 s5, v32, 9
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
@@ -8691,7 +8682,12 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v32, 8
+; GCN-O0-NEXT: v_readlane_b32 s5, v32, 9
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v17
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v17
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -8847,18 +8843,17 @@ define <3 x i32> @insert_dyn_i32_3(<3 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB24_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v4
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v4
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -8988,18 +8983,17 @@ define <3 x float> @insert_dyn_float_3(<3 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB26_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v4
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v4
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9141,12 +9135,6 @@ define <5 x i32> @insert_dyn_i32_5(<5 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB28_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9154,7 +9142,12 @@ define <5 x i32> @insert_dyn_i32_5(<5 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9320,12 +9313,6 @@ define <5 x float> @insert_dyn_float_5(<5 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB30_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9333,7 +9320,12 @@ define <5 x float> @insert_dyn_float_5(<5 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9505,12 +9497,6 @@ define <6 x i32> @insert_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB32_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9519,7 +9505,12 @@ define <6 x i32> @insert_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9703,12 +9694,6 @@ define <6 x float> @insert_dyn_float_6(<6 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB34_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9717,7 +9702,12 @@ define <6 x float> @insert_dyn_float_6(<6 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9907,12 +9897,6 @@ define <7 x i32> @insert_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB36_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9922,7 +9906,12 @@ define <7 x i32> @insert_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v8
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v8
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -10124,12 +10113,6 @@ define <7 x float> @insert_dyn_float_7(<7 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB38_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -10139,7 +10122,12 @@ define <7 x float> @insert_dyn_float_7(<7 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v8
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v8
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
diff --git a/llvm/test/CodeGen/AMDGPU/licm-wwm.mir b/llvm/test/CodeGen/AMDGPU/licm-wwm.mir
index 1ef1bc27095b54..9b19d1b9374703 100644
--- a/llvm/test/CodeGen/AMDGPU/licm-wwm.mir
+++ b/llvm/test/CodeGen/AMDGPU/licm-wwm.mir
@@ -13,7 +13,7 @@ body: |
; GCN: bb.0:
; GCN-NEXT: successors: %bb.1(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_32 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_32 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 1, implicit $exec
; GCN-NEXT: $exec_lo = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
; GCN-NEXT: S_BRANCH %bb.1
@@ -21,7 +21,7 @@ body: |
; GCN-NEXT: bb.1:
; GCN-NEXT: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[ENTER_STRICT_WWM1:%[0-9]+]]:sreg_32 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: [[ENTER_STRICT_WWM1:%[0-9]+]]:sreg_32 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0 = V_READFIRSTLANE_B32 [[V_MOV_B32_e32_]], implicit $exec
; GCN-NEXT: $exec_lo = EXIT_STRICT_WWM [[ENTER_STRICT_WWM1]]
; GCN-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY [[V_READFIRSTLANE_B32_]]
diff --git a/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir b/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir
index 0c9c5ad84c0923..61fe8593ae7c87 100644
--- a/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir
+++ b/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir
@@ -34,7 +34,7 @@ body: |
; REG_ALLOC-NEXT: successors: %bb.3(0x40000000), %bb.5(0x40000000)
; REG_ALLOC-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr4_sgpr5, $sgpr6_sgpr7, $vgpr5_vgpr6_vgpr7_vgpr8:0x0000000000000003, $vgpr6_vgpr7_vgpr8_vgpr9:0x0000000000000003
; REG_ALLOC-NEXT: {{ $}}
- ; REG_ALLOC-NEXT: renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; REG_ALLOC-NEXT: renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
; REG_ALLOC-NEXT: $exec = S_XOR_B64_term $exec, renamable $sgpr2_sgpr3, implicit-def $scc
; REG_ALLOC-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; REG_ALLOC-NEXT: S_BRANCH %bb.3
@@ -72,7 +72,7 @@ body: |
; REG_ALLOC-NEXT: successors: %bb.4(0x80000000)
; REG_ALLOC-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr2_sgpr3, $sgpr4_sgpr5
; REG_ALLOC-NEXT: {{ $}}
- ; REG_ALLOC-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; REG_ALLOC-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; REG_ALLOC-NEXT: S_BRANCH %bb.4
;
; DEAD_INST_DEL-LABEL: name: _amdgpu_cs_main
@@ -103,7 +103,7 @@ body: |
; DEAD_INST_DEL-NEXT: successors: %bb.3(0x40000000), %bb.5(0x40000000)
; DEAD_INST_DEL-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr4_sgpr5, $sgpr6_sgpr7, $vgpr5_vgpr6_vgpr7_vgpr8:0x0000000000000003, $vgpr6_vgpr7_vgpr8_vgpr9:0x0000000000000003
; DEAD_INST_DEL-NEXT: {{ $}}
- ; DEAD_INST_DEL-NEXT: renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; DEAD_INST_DEL-NEXT: renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
; DEAD_INST_DEL-NEXT: $exec = S_XOR_B64_term $exec, renamable $sgpr2_sgpr3, implicit-def $scc
; DEAD_INST_DEL-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; DEAD_INST_DEL-NEXT: S_BRANCH %bb.3
@@ -141,7 +141,7 @@ body: |
; DEAD_INST_DEL-NEXT: successors: %bb.4(0x80000000)
; DEAD_INST_DEL-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr2_sgpr3, $sgpr4_sgpr5
; DEAD_INST_DEL-NEXT: {{ $}}
- ; DEAD_INST_DEL-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; DEAD_INST_DEL-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; DEAD_INST_DEL-NEXT: S_BRANCH %bb.4
bb.0:
successors: %bb.2(0x40000000), %bb.1(0x40000000)
@@ -174,7 +174,7 @@ body: |
bb.1:
successors: %bb.3(0x40000000), %bb.5(0x40000000)
- %17:sreg_64 = S_OR_SAVEEXEC_B64 %15, implicit-def $exec, implicit-def $scc, implicit $exec
+ %17:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %15, implicit-def $exec, implicit-def $scc, implicit $exec
%18:sreg_64_xexec = S_AND_B64 $exec, %17, implicit-def $scc
$exec = S_XOR_B64_term $exec, %18, implicit-def $scc
S_CBRANCH_EXECZ %bb.5, implicit $exec
@@ -207,6 +207,6 @@ body: |
bb.5:
successors: %bb.4(0x80000000)
- $exec = S_OR_B64 $exec, %18, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %18, implicit-def $scc
S_BRANCH %bb.4
...
diff --git a/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir b/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir
index a23fb912893b18..b9c0aa88834171 100644
--- a/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir
+++ b/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir
@@ -36,7 +36,7 @@ body: |
bb.2:
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
S_SETPC_B64_return undef $sgpr30_sgpr31
...
@@ -69,6 +69,6 @@ body: |
bb.2:
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
S_SETPC_B64_return undef $sgpr30_sgpr31
...
diff --git a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir
index 37cd092f0e4db2..e93fed10f84e76 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir
@@ -24,7 +24,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.1:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, %3, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %3, implicit-def $scc
; CHECK-NEXT: S_ENDPGM 0
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
@@ -33,7 +33,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[S_AND_B32_1:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[S_OR_SAVEEXEC_B32_]], implicit-def $scc
; CHECK-NEXT: $exec_lo = S_XOR_B32_term $exec_lo, [[S_AND_B32_1]], implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.1, implicit $exec
@@ -53,7 +53,7 @@ body: |
S_BRANCH %bb.2
bb.1:
- SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
bb.2:
@@ -100,7 +100,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_OR_B32_]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_OR_B32_]], implicit-def $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1(0x80000000)
@@ -121,7 +121,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF killed %2, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -158,7 +158,7 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x80000000)
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.4:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY1]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY1]], implicit-def $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1(0x40000000), %bb.4(0x40000000)
@@ -183,10 +183,10 @@ body: |
bb.3:
successors: %bb.4(0x80000000)
- SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -233,7 +233,7 @@ body: |
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.4(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY1]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY1]], implicit-def $scc
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.6:
@@ -293,7 +293,7 @@ body: |
bb.3:
successors: %bb.5(0x80000000)
- SI_END_CF killed %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.4:
@@ -302,7 +302,7 @@ body: |
bb.5:
successors: %bb.4(0x80000000)
- SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
bb.6:
diff --git a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir
index 32746ef5fce9e8..2b364b3ea63417 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir
@@ -32,7 +32,7 @@ body: |
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.3(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY4:%[0-9]+]]:sreg_64_xexec = COPY killed [[S_MOV_B64_term]]
+ ; CHECK-NEXT: [[COPY4:%[0-9]+]]:sreg_64_xexec = bb-prolog COPY killed [[S_MOV_B64_term]]
; CHECK-NEXT: $exec = S_OR_B64_term $exec, killed [[COPY4]], implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
@@ -72,7 +72,7 @@ body: |
%4:sreg_64_xexec = PHI %5, %bb.2, %3, %bb.0
%6:vgpr_32 = PHI %7, %bb.2, %1, %bb.0
- SI_END_CF killed %4, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %4, implicit-def $exec, implicit-def dead $scc, implicit $exec
%8:vgpr_32 = nsw V_ADD_U32_e32 1, killed %6, implicit $exec
bb.2:
diff --git a/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir b/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir
index 84b348fd86bb0c..c8ced4bd5e1c0c 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir
@@ -110,7 +110,7 @@ body: |
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 %2, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %2, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sreg_64_xexec = COPY $sgpr4_sgpr5
; CHECK-NEXT: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 0, [[COPY]], implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir b/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir
index 8f9468cf5b3001..2203bde7123616 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir
@@ -120,7 +120,7 @@ body: |
%24:vgpr_32 = PHI %17, %bb.1, %21, %bb.2
%25:vreg_1 = PHI %7, %bb.1, %23, %bb.2
- SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%26:sreg_32 = S_MOV_B32 -1
%27:sreg_32 = IMPLICIT_DEF
%28:sreg_32 = COPY %25
@@ -143,7 +143,7 @@ body: |
%18:vgpr_32 = PHI %29, %bb.3, %24, %bb.4
%16:sreg_32 = PHI %27, %bb.3, %33, %bb.4
%36:vreg_1 = PHI %30, %bb.3, %35, %bb.4
- SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%37:sreg_32 = COPY %36
%14:sreg_32 = SI_IF_BREAK %37, %13, implicit-def dead $scc
SI_LOOP %14, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -151,7 +151,7 @@ body: |
bb.6:
%38:sreg_32 = PHI %14, %bb.5
- SI_END_CF %38, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %38, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir
index 2abcb46977827e..06778fba76f370 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir
@@ -95,7 +95,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x7c000000), %bb.5(0x04000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
; CHECK-NEXT: %loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
; CHECK-NEXT: S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -172,7 +172,7 @@ body: |
bb.3:
successors: %bb.4(0x7c000000), %bb.5(0x04000000)
- $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
%loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -260,7 +260,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x7c000000), %bb.5(0x04000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
; CHECK-NEXT: %loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
; CHECK-NEXT: S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -337,7 +337,7 @@ body: |
bb.3:
successors: %bb.4(0x7c000000), %bb.5(0x04000000)
- $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
%loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -416,7 +416,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x7c000000), %bb.5(0x04000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
; CHECK-NEXT: %loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
; CHECK-NEXT: S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -501,7 +501,7 @@ body: |
bb.3:
successors: %bb.4(0x7c000000), %bb.5(0x04000000)
- $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
%loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.5, implicit killed $scc
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
index 8c9e4a5a26c831..f02efdc9881a3d 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
@@ -1080,7 +1080,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -1160,7 +1160,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -1237,7 +1237,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4186,7 +4186,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4267,7 +4267,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4344,7 +4344,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4433,7 +4433,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4520,7 +4520,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4603,7 +4603,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4698,7 +4698,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4791,7 +4791,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4880,7 +4880,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4981,7 +4981,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -5080,7 +5080,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -5175,7 +5175,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -5824,7 +5824,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -5965,7 +5965,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -6102,7 +6102,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -6267,7 +6267,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -6438,7 +6438,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -6605,7 +6605,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -6824,7 +6824,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -7061,7 +7061,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -7294,7 +7294,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir
index b1bcd89e06c283..fd6896d1da6f24 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir
@@ -80,7 +80,7 @@ body: |
bb.2:
%22:vgpr_32 = PHI %3, %bb.0, %20, %bb.1
%23:vgpr_32 = PHI %4, %bb.0, %21, %bb.1
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
S_ENDPGM 0, implicit %22, implicit %23
@@ -163,7 +163,7 @@ body: |
bb.2:
%22:vgpr_32 = PHI %3, %bb.0, %20, %bb.1
%23:vgpr_32 = PHI %4, %bb.0, %21, %bb.1
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
%24:vgpr_32 = V_ADD_F32_e32 %14, %11, implicit $mode, implicit $exec
@@ -248,7 +248,7 @@ body: |
bb.2:
%22:vgpr_32 = PHI %3, %bb.0, %20, %bb.1
%23:vgpr_32 = PHI %4, %bb.0, %21, %bb.1
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
S_ENDPGM 0, implicit %22, implicit %23
@@ -550,7 +550,7 @@ body: |
successors: %bb.4(0x40000000), %bb.6(0x40000000)
liveins: $vgpr0, $vgpr1, $vgpr2_vgpr3, $vcc
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_CBRANCH_EXECZ %bb.6, implicit $exec
bb.4:
@@ -703,7 +703,7 @@ body: |
successors: %bb.5(0x40000000), %bb.7(0x40000000)
liveins: $vgpr0, $vgpr1, $vgpr2_vgpr3, $vcc
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_CBRANCH_EXECZ %bb.7, implicit $exec
S_BRANCH %bb.5
@@ -795,7 +795,7 @@ body: |
successors: %bb.3(0x80000000)
%13:vgpr_32 = PHI %1, %bb.0, %12, %bb.1
- SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
S_ENDPGM 0, implicit %13
@@ -845,7 +845,7 @@ body: |
bb.2:
%7:vgpr_32 = PHI %1, %bb.0, %6, %bb.1
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit %7
...
---
@@ -893,6 +893,6 @@ body: |
bb.2:
%7:vgpr_32 = PHI %1, %bb.0, %6, %bb.1
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit %7
...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir
index 6b606fbf0847b4..abb7f80f1e42f8 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir
@@ -108,7 +108,7 @@ body: |
S_BRANCH %bb.4
bb.3:
- SI_END_CF %12, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %12, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
bb.4:
@@ -128,7 +128,7 @@ body: |
%10:sreg_32 = PHI %45, %bb.2, %46, %bb.4
%8:sreg_32 = PHI %39, %bb.2, %40, %bb.4
%9:vgpr_32 = PHI %36, %bb.2, %6, %bb.4
- SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%11:sreg_32 = SI_IF_BREAK %10, %2, implicit-def dead $scc
%12:sreg_32 = SI_IF_BREAK %8, %0, implicit-def dead $scc
SI_LOOP %11, %bb.2, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -138,7 +138,7 @@ body: |
successors: %bb.3(0x04000000), %bb.1(0x7c000000)
%13:vgpr_32 = PHI %9, %bb.5
- SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
SI_LOOP %12, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.3
...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir
index 574bfaef981be8..dde7fe76074be9 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir
@@ -96,7 +96,7 @@ body: |
bb.5:
%7:vgpr_32 = PHI %0, %bb.4
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
INLINEASM &"", 1, implicit %5
S_BRANCH %bb.2
@@ -207,7 +207,7 @@ body: |
bb.5:
%7:vgpr_32 = PHI %0, %bb.4
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
INLINEASM &"", 1, implicit %5
S_BRANCH %bb.2
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir
index c8afebf22e08d3..70d6d2a3bd60d0 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir
@@ -32,7 +32,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: bb-prolog SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -59,7 +59,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
FLAT_STORE_DWORD %3, %9, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
@@ -95,7 +95,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: bb-prolog SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -124,7 +124,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
FLAT_STORE_DWORD %3, %11, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
@@ -158,7 +158,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: bb-prolog SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -185,7 +185,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
FLAT_STORE_DWORD %3, %11, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
@@ -214,7 +214,7 @@ body: |
; CHECK-NEXT: SI_WATERFALL_LOOP %bb.1, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec_lo = S_MOV_B32 [[S_MOV_B32_]]
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_MOV_B32 [[S_MOV_B32_]]
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -236,7 +236,7 @@ body: |
SI_WATERFALL_LOOP %bb.1, implicit $exec
bb.2:
- $exec_lo = S_MOV_B32 %2
+ $exec_lo = bb-prolog S_MOV_B32 %2
FLAT_STORE_DWORD %1, %6, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir b/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir
index f171652bf7b555..3abf41009aa4d0 100644
--- a/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir
+++ b/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir
@@ -105,7 +105,7 @@ body: |
; CHECK-NEXT: bb.2.exit:
; CHECK-NEXT: liveins: $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0 (%ir-block.0):
successors: %bb.1.atomic(0x40000000), %bb.2.exit(0x40000000)
@@ -140,7 +140,7 @@ body: |
bb.2.exit:
liveins: $sgpr2_sgpr3
- $exec = S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/merge-m0.mir b/llvm/test/CodeGen/AMDGPU/merge-m0.mir
index a06ffafa7a046a..11d87bd043749d 100644
--- a/llvm/test/CodeGen/AMDGPU/merge-m0.mir
+++ b/llvm/test/CodeGen/AMDGPU/merge-m0.mir
@@ -204,7 +204,7 @@ body: |
successors: %bb.1, %bb.2
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
%0 = IMPLICIT_DEF
%1 = IMPLICIT_DEF
S_CBRANCH_VCCZ %bb.1, implicit undef $vcc
@@ -242,7 +242,7 @@ body: |
...
# GCN-LABEL: name: move-m0-with-prologue
-# GCN: $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+# GCN: $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
# GCN: $m0 = S_MOV_B32 -1
# GCN-NEXT: $vgpr0 = V_MOV_B32_e32 0, implicit $exec
# GCN-NEXT: DS_GWS_INIT $vgpr0, 0, implicit $m0, implicit $exec
@@ -252,7 +252,7 @@ body: |
bb.0:
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$vgpr0 = V_MOV_B32_e32 0, implicit $exec
$m0 = S_MOV_B32 -1
DS_GWS_INIT $vgpr0, 0, implicit $m0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
index c97acc3d55de4a..5f30a47e7dbf6e 100644
--- a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
+++ b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
@@ -164,23 +164,23 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB0_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -190,6 +190,7 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v7, s8, 3
; W64-O0-NEXT: v_writelane_b32 v7, s9, 4
; W64-O0-NEXT: v_writelane_b32 v7, s10, 5
@@ -201,6 +202,7 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB0_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -212,15 +214,14 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s10, v7, 5
; W64-O0-NEXT: v_readlane_b32 s11, v7, 6
; W64-O0-NEXT: v_readlane_b32 s6, v7, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB0_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -228,7 +229,6 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s4, v7, 1
; W64-O0-NEXT: v_readlane_b32 s5, v7, 2
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_xor_saveexec_b64 s[4:5], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
@@ -534,23 +534,23 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -560,6 +560,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 3
; W64-O0-NEXT: v_writelane_b32 v17, s9, 4
; W64-O0-NEXT: v_writelane_b32 v17, s10, 5
@@ -571,6 +572,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB1_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -582,9 +584,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: v_readlane_b32 s10, v17, 5
; W64-O0-NEXT: v_readlane_b32 s11, v17, 6
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
@@ -605,23 +605,23 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_4: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -631,6 +631,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 11
; W64-O0-NEXT: v_writelane_b32 v17, s9, 12
; W64-O0-NEXT: v_writelane_b32 v17, s10, 13
@@ -642,6 +643,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.5: ; in Loop: Header=BB1_4 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -653,29 +655,26 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: v_readlane_b32 s10, v17, 13
; W64-O0-NEXT: v_readlane_b32 s11, v17, 14
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB1_4
; W64-O0-NEXT: ; %bb.6:
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
-; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
-; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
+; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
+; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: global_store_dword v[3:4], v5, off
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: global_store_dword v[0:1], v2, off
@@ -1041,23 +1040,23 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1067,6 +1066,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 4
; W64-O0-NEXT: v_writelane_b32 v13, s9, 5
; W64-O0-NEXT: v_writelane_b32 v13, s10, 6
@@ -1078,6 +1078,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB2_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1089,27 +1090,24 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: v_readlane_b32 s10, v13, 6
; W64-O0-NEXT: v_readlane_b32 s11, v13, 7
; W64-O0-NEXT: v_readlane_b32 s6, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
; W64-O0-NEXT: v_readlane_b32 s6, v13, 2
; W64-O0-NEXT: v_readlane_b32 s7, v13, 3
; W64-O0-NEXT: s_mov_b64 exec, s[6:7]
-; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b32 s5, 0x3ff
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_and_b32_e64 v1, v1, s5
; W64-O0-NEXT: v_cmp_eq_u32_e64 s[6:7], v1, s4
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
@@ -1139,23 +1137,23 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_5: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1165,6 +1163,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 15
; W64-O0-NEXT: v_writelane_b32 v13, s9, 16
; W64-O0-NEXT: v_writelane_b32 v13, s10, 17
@@ -1176,6 +1175,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.6: ; in Loop: Header=BB2_5 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1187,15 +1187,14 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: v_readlane_b32 s10, v13, 17
; W64-O0-NEXT: v_readlane_b32 s11, v13, 18
; W64-O0-NEXT: v_readlane_b32 s6, v13, 12
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_5
; W64-O0-NEXT: ; %bb.7:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1203,8 +1202,6 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: v_readlane_b32 s4, v13, 13
; W64-O0-NEXT: v_readlane_b32 s5, v13, 14
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; W64-O0-NEXT: .LBB2_8: ; %bb2
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
diff --git a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
index 831d876e280a2e..ca2d277e2b9b53 100644
--- a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
+++ b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
@@ -170,23 +170,23 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB0_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -196,6 +196,7 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v7, s8, 3
; W64-O0-NEXT: v_writelane_b32 v7, s9, 4
; W64-O0-NEXT: v_writelane_b32 v7, s10, 5
@@ -207,6 +208,7 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB0_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -218,15 +220,14 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s10, v7, 5
; W64-O0-NEXT: v_readlane_b32 s11, v7, 6
; W64-O0-NEXT: v_readlane_b32 s6, v7, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB0_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -234,7 +235,6 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s4, v7, 1
; W64-O0-NEXT: v_readlane_b32 s5, v7, 2
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_xor_saveexec_b64 s[4:5], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
@@ -554,23 +554,23 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -580,6 +580,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 3
; W64-O0-NEXT: v_writelane_b32 v17, s9, 4
; W64-O0-NEXT: v_writelane_b32 v17, s10, 5
@@ -591,6 +592,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB1_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -602,9 +604,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: v_readlane_b32 s10, v17, 5
; W64-O0-NEXT: v_readlane_b32 s11, v17, 6
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
@@ -625,23 +625,23 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_4: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -651,6 +651,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 11
; W64-O0-NEXT: v_writelane_b32 v17, s9, 12
; W64-O0-NEXT: v_writelane_b32 v17, s10, 13
@@ -662,6 +663,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.5: ; in Loop: Header=BB1_4 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -673,29 +675,26 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: v_readlane_b32 s10, v17, 13
; W64-O0-NEXT: v_readlane_b32 s11, v17, 14
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB1_4
; W64-O0-NEXT: ; %bb.6:
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
-; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
-; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
+; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
+; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: global_store_dword v[3:4], v5, off
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: global_store_dword v[0:1], v2, off
@@ -1072,23 +1071,23 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1098,6 +1097,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 4
; W64-O0-NEXT: v_writelane_b32 v13, s9, 5
; W64-O0-NEXT: v_writelane_b32 v13, s10, 6
@@ -1109,6 +1109,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB2_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1120,27 +1121,24 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: v_readlane_b32 s10, v13, 6
; W64-O0-NEXT: v_readlane_b32 s11, v13, 7
; W64-O0-NEXT: v_readlane_b32 s6, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
; W64-O0-NEXT: v_readlane_b32 s6, v13, 2
; W64-O0-NEXT: v_readlane_b32 s7, v13, 3
; W64-O0-NEXT: s_mov_b64 exec, s[6:7]
-; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b32 s5, 0x3ff
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_and_b32_e64 v1, v1, s5
; W64-O0-NEXT: v_cmp_eq_u32_e64 s[6:7], v1, s4
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
@@ -1154,16 +1152,15 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execz .LBB2_8
; W64-O0-NEXT: ; %bb.4: ; %bb1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: v_readlane_b32 s4, v13, 0
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v13, 0
; W64-O0-NEXT: v_mov_b32_e32 v6, v4
; W64-O0-NEXT: v_mov_b32_e32 v0, v3
; W64-O0-NEXT: v_mov_b32_e32 v4, v2
@@ -1188,23 +1185,23 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_5: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1214,6 +1211,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 15
; W64-O0-NEXT: v_writelane_b32 v13, s9, 16
; W64-O0-NEXT: v_writelane_b32 v13, s10, 17
@@ -1225,6 +1223,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.6: ; in Loop: Header=BB2_5 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1236,15 +1235,14 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: v_readlane_b32 s10, v13, 17
; W64-O0-NEXT: v_readlane_b32 s11, v13, 18
; W64-O0-NEXT: v_readlane_b32 s6, v13, 12
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_5
; W64-O0-NEXT: ; %bb.7:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1252,8 +1250,6 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: v_readlane_b32 s4, v13, 13
; W64-O0-NEXT: v_readlane_b32 s5, v13, 14
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; W64-O0-NEXT: .LBB2_8: ; %bb2
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
diff --git a/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir b/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir
index c2e24fcd6de067..177ad8ab6d308e 100644
--- a/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir
+++ b/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir
@@ -66,7 +66,7 @@ body: |
; GFX9-NEXT: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; GFX9-NEXT: {{ $}}
; GFX9-NEXT: bb.2:
- ; GFX9-NEXT: $exec = S_MOV_B64 [[S_MOV_B64_]]
+ ; GFX9-NEXT: $exec = bb-prolog S_MOV_B64 [[S_MOV_B64_]]
; GFX9-NEXT: $sgpr30_sgpr31 = COPY [[COPY]]
; GFX9-NEXT: $vgpr0 = COPY [[V_MOV_B32_e32_]]
; GFX9-NEXT: S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0, implicit undef $vgpr1, implicit undef $vgpr2, implicit undef $vgpr3
@@ -109,7 +109,7 @@ body: |
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.2:
- $exec = S_MOV_B64 killed %20
+ $exec = bb-prolog S_MOV_B64 killed %20
$sgpr30_sgpr31 = COPY killed %17
$vgpr0 = COPY killed %24
S_SETPC_B64_return killed $sgpr30_sgpr31, implicit killed $vgpr0, implicit undef $vgpr1, implicit undef $vgpr2, implicit undef $vgpr3
diff --git a/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir b/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir
index 6b3ef6c3b4547e..10dde8238bf476 100644
--- a/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir
+++ b/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir
@@ -160,7 +160,7 @@ body: |
BUFFER_STORE_DWORD_ADDR64 killed %29, killed %30, killed %28, 0, 0, 0, 0, implicit $exec
bb.2.bb2:
- SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -310,7 +310,7 @@ body: |
BUFFER_STORE_DWORD_ADDR64 killed %38, killed %39, killed %37, 0, 0, 0, 0, implicit $exec
bb.2.bb2:
- SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -433,7 +433,7 @@ body: |
BUFFER_STORE_DWORD_ADDR64 killed %29, killed %30, killed %28, 0, 0, 0, 0, implicit $exec
bb.2.bb2:
- SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir
index df83b74042ad26..e9e08ece38faf5 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir
@@ -283,11 +283,11 @@ body: |
S_BRANCH %bb.18
bb.18:
- $exec_lo = S_OR_B32 $exec_lo, %65, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %65, implicit-def $scc
S_ENDPGM 0
bb.19:
- $exec_lo = S_OR_B32 $exec_lo, %59, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %59, implicit-def $scc
S_BRANCH %bb.2
...
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir
index 391f20b241c3b8..12d1abdcc720f6 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir
@@ -21,23 +21,23 @@ body: |
; GCN-NEXT: {{ $}}
; GCN-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[DEF]], implicit-def dead $scc
; GCN-NEXT: $vcc_lo = COPY [[S_AND_B32_]]
- ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.6, implicit $vcc
+ ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.6, implicit $vcc_lo
; GCN-NEXT: S_BRANCH %bb.1
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.2:
; GCN-NEXT: successors: %bb.3(0x80000000)
; GCN-NEXT: {{ $}}
; GCN-NEXT: $vcc_lo = S_ANDN2_B32 $exec_lo, %8, implicit-def dead $scc
- ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.3, implicit $vcc
+ ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.3, implicit $vcc_lo
; GCN-NEXT: S_BRANCH %bb.3
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x50000000), %bb.5(0x30000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec_lo = S_OR_B32 $exec_lo, %6, implicit-def $scc
+ ; GCN-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %6, implicit-def $scc
; GCN-NEXT: [[V_CMP_NE_U32_e64_:%[0-9]+]]:sreg_32_xm0_xexec = V_CMP_NE_U32_e64 1, %5, implicit $exec
; GCN-NEXT: $vcc_lo = S_AND_B32 $exec_lo, [[V_CMP_NE_U32_e64_]], implicit-def dead $scc
- ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.5, implicit $vcc
+ ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.5, implicit $vcc_lo
; GCN-NEXT: S_BRANCH %bb.4
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
@@ -81,7 +81,7 @@ body: |
bb.3:
successors: %bb.4(0x50000000), %bb.5(0x30000000)
- $exec_lo = S_OR_B32 $exec_lo, %6, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %6, implicit-def $scc
%7:sreg_32_xm0_xexec = V_CMP_NE_U32_e64 1, %5, implicit $exec
$vcc_lo = S_AND_B32 $exec_lo, %7, implicit-def dead $scc
S_CBRANCH_VCCNZ %bb.5, implicit killed $vcc
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir
index 2d0ec54c5940f8..414587d3d34e00 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir
@@ -136,7 +136,7 @@ body: |
S_BRANCH %bb.21
bb.20:
- $exec_lo = S_OR_B32 $exec_lo, %34, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %34, implicit-def $scc
S_BRANCH %bb.30
bb.21:
@@ -184,7 +184,7 @@ body: |
S_BRANCH %bb.31
bb.30:
- $exec_lo = S_OR_B32 $exec_lo, %32, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %32, implicit-def $scc
S_BRANCH %bb.10
bb.31:
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir
index 1280f015358fe6..b01ec7aa78a40e 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir
@@ -32,8 +32,8 @@ body: |
; GCN-NEXT: bb.2:
; GCN-NEXT: successors: %bb.3(0x40000000), %bb.6(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
- ; GCN-NEXT: $exec = S_AND_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: $exec = bb-prolog S_AND_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: [[S_AND_B64_1:%[0-9]+]]:sreg_64 = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def $scc
; GCN-NEXT: $exec = S_XOR_B64_term $exec, [[S_AND_B64_1]], implicit-def $scc
; GCN-NEXT: S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -55,10 +55,10 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.6:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def $scc
bb.0:
successors: %bb.1, %bb.2
liveins: $vgpr0
@@ -78,8 +78,8 @@ body: |
bb.2:
successors: %bb.3, %bb.6
- %6:sreg_64 = S_OR_SAVEEXEC_B64 %5, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_AND_B64 $exec, %0, implicit-def dead $scc
+ %6:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %5, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_AND_B64 $exec, %0, implicit-def dead $scc
%7:sreg_64 = S_AND_B64 $exec, %6, implicit-def $scc
$exec = S_XOR_B64_term $exec, %7, implicit-def $scc
S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -98,10 +98,10 @@ body: |
bb.4:
bb.5:
- $exec = S_OR_B64 $exec, %9, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %9, implicit-def $scc
bb.6:
- $exec = S_OR_B64 $exec, %7, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %7, implicit-def $scc
...
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir b/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir
index f7e9e248c7698b..499f457a570eae 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir
@@ -156,7 +156,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -193,7 +193,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -230,7 +230,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -271,7 +271,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -309,7 +309,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1, $sgpr4_sgpr5_sgpr6_sgpr7
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr4_sgpr5_sgpr6_sgpr7, 0, 0, 0, 0, implicit $exec
@@ -351,7 +351,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr0 = S_MOV_B32 0
$sgpr1 = S_MOV_B32 1
$sgpr2 = S_MOV_B32 -1
@@ -392,7 +392,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -431,7 +431,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -468,7 +468,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -505,7 +505,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -544,7 +544,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll b/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll
index 21d723018c729a..250803fda00396 100644
--- a/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll
+++ b/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll
@@ -231,12 +231,15 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: s_cbranch_vccnz .LBB0_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v0, off, s[92:95], 0 offset:4 ; 4-byte Folded Reload
+; GCN-NEXT: buffer_load_dword v2, off, s[92:95], 0 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
; GCN-NEXT: buffer_load_dword v1, off, s[92:95], 0 offset:8 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
-; GCN-NEXT: s_waitcnt vmcnt(1)
+; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
+; GCN-NEXT: buffer_load_dword v0, off, s[92:95], 0 offset:4 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[34:35]
+; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s8, v0, 56
; GCN-NEXT: v_readlane_b32 s9, v0, 57
; GCN-NEXT: v_readlane_b32 s10, v0, 58
@@ -301,7 +304,6 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: v_readlane_b32 s73, v0, 5
; GCN-NEXT: v_readlane_b32 s74, v0, 6
; GCN-NEXT: v_readlane_b32 s75, v0, 7
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s76, v1, 56
; GCN-NEXT: v_readlane_b32 s77, v1, 57
; GCN-NEXT: v_readlane_b32 s78, v1, 58
@@ -326,9 +328,6 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: v_readlane_b32 s5, v1, 5
; GCN-NEXT: v_readlane_b32 s6, v1, 6
; GCN-NEXT: v_readlane_b32 s7, v1, 7
-; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v2, off, s[92:95], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[0:7]
; GCN-NEXT: ;;#ASMEND
@@ -387,7 +386,6 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[0:7]
; GCN-NEXT: ;;#ASMEND
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s0, v2, 0
; GCN-NEXT: v_readlane_b32 s1, v2, 1
; GCN-NEXT: v_readlane_b32 s2, v2, 2
@@ -601,12 +599,12 @@ define amdgpu_kernel void @split_sgpr_spill_2_vgprs(ptr addrspace(1) %out, i32 %
; GCN-NEXT: s_cbranch_vccnz .LBB1_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[28:29], -1
-; GCN-NEXT: buffer_load_dword v0, off, s[52:55], 0 ; 4-byte Folded Reload
+; GCN-NEXT: buffer_load_dword v1, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[28:29]
; GCN-NEXT: s_or_saveexec_b64 s[28:29], -1
-; GCN-NEXT: buffer_load_dword v1, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
+; GCN-NEXT: buffer_load_dword v0, off, s[52:55], 0 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[28:29]
-; GCN-NEXT: s_waitcnt vmcnt(1)
+; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s16, v0, 8
; GCN-NEXT: v_readlane_b32 s17, v0, 9
; GCN-NEXT: v_readlane_b32 s20, v0, 0
@@ -617,7 +615,6 @@ define amdgpu_kernel void @split_sgpr_spill_2_vgprs(ptr addrspace(1) %out, i32 %
; GCN-NEXT: v_readlane_b32 s25, v0, 5
; GCN-NEXT: v_readlane_b32 s26, v0, 6
; GCN-NEXT: v_readlane_b32 s27, v0, 7
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s36, v1, 32
; GCN-NEXT: v_readlane_b32 s37, v1, 33
; GCN-NEXT: v_readlane_b32 s38, v1, 34
@@ -853,6 +850,9 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill(ptr addrspace(1) %out, i32 %
; GCN-NEXT: s_cbranch_vccnz .LBB2_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
+; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[34:35]
+; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
; GCN-NEXT: buffer_load_dword v31, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: s_waitcnt vmcnt(0)
@@ -904,9 +904,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill(ptr addrspace(1) %out, i32 %
; GCN-NEXT: v_readlane_b32 s29, v31, 13
; GCN-NEXT: v_readlane_b32 s30, v31, 14
; GCN-NEXT: v_readlane_b32 s31, v31, 15
-; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[16:31]
; GCN-NEXT: ;;#ASMEND
@@ -929,7 +926,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill(ptr addrspace(1) %out, i32 %
; GCN-NEXT: v_readlane_b32 s17, v31, 61
; GCN-NEXT: v_readlane_b32 s18, v31, 62
; GCN-NEXT: v_readlane_b32 s19, v31, 63
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s0, v32, 0
; GCN-NEXT: v_readlane_b32 s1, v32, 1
; GCN-NEXT: ;;#ASMSTART
@@ -1095,6 +1091,9 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill_live_v0(i32 %in) #1 {
; GCN-NEXT: s_cbranch_vccnz .LBB3_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
+; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[34:35]
+; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
; GCN-NEXT: buffer_load_dword v31, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: s_waitcnt vmcnt(0)
@@ -1146,9 +1145,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill_live_v0(i32 %in) #1 {
; GCN-NEXT: v_readlane_b32 s29, v31, 13
; GCN-NEXT: v_readlane_b32 s30, v31, 14
; GCN-NEXT: v_readlane_b32 s31, v31, 15
-; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; def v0
; GCN-NEXT: ;;#ASMEND
@@ -1174,7 +1170,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill_live_v0(i32 %in) #1 {
; GCN-NEXT: v_readlane_b32 s17, v31, 61
; GCN-NEXT: v_readlane_b32 s18, v31, 62
; GCN-NEXT: v_readlane_b32 s19, v31, 63
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s0, v32, 0
; GCN-NEXT: v_readlane_b32 s1, v32, 1
; GCN-NEXT: ;;#ASMSTART
diff --git a/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir b/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir
index c2b9f34ea78b0e..dde94836c5e952 100644
--- a/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir
+++ b/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir
@@ -8,8 +8,8 @@
# CHECK: [[IF_INPUT_REG:%[0-9]+]]:sreg_64 = S_MOV_B64_term killed [[IF_SOURCE0]], implicit $exec
# CHECK: bb.1:
-# CHECK: [[END_CF_ARG:%[0-9]+]]:sreg_64 = COPY killed [[IF_INPUT_REG]]
-# CHECK: SI_END_CF killed [[END_CF_ARG]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+# CHECK: [[END_CF_ARG:%[0-9]+]]:sreg_64 = bb-prolog COPY killed [[IF_INPUT_REG]]
+# CHECK: bb-prolog SI_END_CF killed [[END_CF_ARG]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
# CHECK: bb.2:
# CHECK: [[IF_SOURCE1:%[0-9]+]]:sreg_64 = SI_IF [[COND]], %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -37,7 +37,7 @@ body: |
%24:sreg_64 = PHI %20, %bb.3, %22, %bb.0
%23:vgpr_32 = PHI %19, %bb.3, %18, %bb.0
- SI_END_CF %24, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %24, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%3:vgpr_32, dead %10:sreg_64 = nsw V_ADD_CO_U32_e64 1, %23, 0, implicit $exec
bb.3:
diff --git a/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir b/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir
index ce2ede3fe215ee..674b3a93f07c5e 100644
--- a/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir
+++ b/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir
@@ -15,8 +15,6 @@ body: |
; CHECK-NEXT: liveins: $sgpr0, $sgpr3, $sgpr2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vgpr1 = IMPLICIT_DEF
- ; CHECK-NEXT: renamable $sgpr4 = COPY $sgpr2
- ; CHECK-NEXT: renamable $sgpr2 = COPY $sgpr3
; CHECK-NEXT: $vgpr1 = SI_SPILL_S32_TO_VGPR $sgpr0, 0, $vgpr1
; CHECK-NEXT: $sgpr1 = S_AND_SAVEEXEC_B32 $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.1, implicit $exec
@@ -27,8 +25,10 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.1(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr0, $sgpr2, $sgpr4, $vgpr1
+ ; CHECK-NEXT: liveins: $sgpr0, $sgpr2, $sgpr3, $vgpr1
; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: renamable $sgpr4 = COPY $sgpr2
+ ; CHECK-NEXT: renamable $sgpr2 = COPY $sgpr3
; CHECK-NEXT: $sgpr3 = SI_RESTORE_S32_FROM_VGPR $vgpr1, 0
; CHECK-NEXT: renamable $sgpr0_sgpr1 = S_GETPC_B64_pseudo
; CHECK-NEXT: renamable $sgpr5 = COPY $sgpr1
diff --git a/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir b/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir
index 794d96dfa2365a..a122e0abc6a6d4 100644
--- a/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir
+++ b/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir
@@ -41,10 +41,8 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $vgpr0, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9, $sgpr10_sgpr11
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: SI_SPILL_S32_SAVE $sgpr15, %stack.3, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.3, addrspace 5)
- ; CHECK-NEXT: SI_SPILL_S32_SAVE $sgpr14, %stack.4, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.4, addrspace 5)
; CHECK-NEXT: renamable $sgpr12_sgpr13 = V_CMP_GT_I32_e64 1, undef %18:vgpr_32, implicit $exec
- ; CHECK-NEXT: renamable $sgpr14_sgpr15 = V_CMP_EQ_U32_e64 0, undef %18:vgpr_32, implicit $exec
+ ; CHECK-NEXT: renamable $sgpr74_sgpr75 = V_CMP_EQ_U32_e64 0, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr72_sgpr73 = V_CMP_NE_U32_e64 0, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr18_sgpr19 = V_CMP_GT_I32_e64 0, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr52 = S_MOV_B32 0
@@ -52,12 +50,12 @@ body: |
; CHECK-NEXT: [[COPY:%[0-9]+]]:vreg_1024_align2 = COPY renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, implicit $exec
; CHECK-NEXT: renamable $sgpr22_sgpr23 = V_CMP_NE_U32_e64 1, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr53 = S_MOV_B32 1083786240
- ; CHECK-NEXT: SI_SPILL_S1024_SAVE renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, %stack.5, implicit $exec, implicit $sgpr32 :: (store (s1024) into %stack.5, align 4, addrspace 5)
+ ; CHECK-NEXT: SI_SPILL_S1024_SAVE renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, %stack.4, implicit $exec, implicit $sgpr32 :: (store (s1024) into %stack.4, align 4, addrspace 5)
; CHECK-NEXT: S_BRANCH %bb.1
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr22_sgpr23, implicit-def dead $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vreg_1024_align2 = COPY [[COPY]]
@@ -66,9 +64,9 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.11(0x40000000), %bb.5(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.5, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.5, align 4, addrspace 5)
+ ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.4, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr68_sgpr69 = lr-split COPY killed renamable $sgpr52_sgpr53
; CHECK-NEXT: renamable $sgpr36 = COPY renamable $sgpr68
; CHECK-NEXT: renamable $sgpr37 = COPY renamable $sgpr68
@@ -108,28 +106,30 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr60 = lr-split COPY killed renamable $sgpr16
+ ; CHECK-NEXT: renamable $sgpr60 = lr-split COPY killed renamable $sgpr14
+ ; CHECK-NEXT: renamable $sgpr62 = lr-split COPY killed renamable $sgpr15
+ ; CHECK-NEXT: SI_SPILL_S32_SAVE killed renamable $sgpr16, %stack.3, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.3, addrspace 5)
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr4_sgpr5, 0, CustomRegMask($sgpr60,$sgpr62)
; CHECK-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.4:
; CHECK-NEXT: successors: %bb.17(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr60
+ ; CHECK-NEXT: liveins: $sgpr60, $sgpr62
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
- ; CHECK-NEXT: $sgpr12 = SI_SPILL_S32_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.4, addrspace 5)
- ; CHECK-NEXT: $sgpr13 = SI_SPILL_S32_RESTORE %stack.3, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.3, addrspace 5)
- ; CHECK-NEXT: $sgpr14 = COPY killed renamable $sgpr60
+ ; CHECK-NEXT: $sgpr12 = COPY killed renamable $sgpr60
+ ; CHECK-NEXT: $sgpr13 = COPY killed renamable $sgpr62
+ ; CHECK-NEXT: $sgpr14 = SI_SPILL_S32_RESTORE %stack.3, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.3, addrspace 5)
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr4_sgpr5, 0, csr_amdgpu_noregs, implicit $sgpr12, implicit $sgpr13, implicit $sgpr14
; CHECK-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: S_BRANCH %bb.17
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.12(0x40000000), %bb.6(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $sgpr26_sgpr27 = S_AND_B64 renamable $sgpr72_sgpr73, undef renamable $sgpr64_sgpr65, implicit-def dead $scc
; CHECK-NEXT: renamable $sgpr24_sgpr25 = V_CMP_GT_I32_e64 0, undef %18:vgpr_32, implicit $exec
@@ -138,39 +138,39 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.6:
; CHECK-NEXT: successors: %bb.7(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: dead [[V_CNDMASK_B32_e64_:%[0-9]+]]:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, $sgpr18_sgpr19, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.7:
; CHECK-NEXT: successors: %bb.8(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $sgpr34_sgpr35 = nofpexcept V_CMP_NLT_F64_e64 0, undef $sgpr4_sgpr5, 0, undef %29:vreg_64_align2, 0, implicit $mode, implicit $exec
; CHECK-NEXT: renamable $sgpr68_sgpr69 = nofpexcept V_CMP_NLT_F64_e64 0, 4607182418800017408, 0, undef %29:vreg_64_align2, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: dead [[V_INDIRECT_REG_READ_GPR_IDX_B32_V32_:%[0-9]+]]:vgpr_32 = V_INDIRECT_REG_READ_GPR_IDX_B32_V32 [[COPY1]], undef $sgpr4, 11, implicit-def $m0, implicit $m0, implicit $exec
+ ; CHECK-NEXT: dead [[V_INDIRECT_REG_READ_GPR_IDX_B32_V32_:%[0-9]+]]:vgpr_32 = V_INDIRECT_REG_READ_GPR_IDX_B32_V32 [[COPY1]], undef $sgpr14, 11, implicit-def $m0, implicit $m0, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.8:
; CHECK-NEXT: successors: %bb.10(0x40000000), %bb.9(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr34_sgpr35, implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_VCCNZ %bb.10, implicit $vcc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.9:
; CHECK-NEXT: successors: %bb.10(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.5, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.5, align 4, addrspace 5)
+ ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.4, align 4, addrspace 5)
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vreg_64_align2 = COPY killed renamable $sgpr52_sgpr53, implicit $exec
; CHECK-NEXT: GLOBAL_STORE_DWORDX2_SADDR undef %18:vgpr_32, [[COPY2]], undef renamable $sgpr4_sgpr5, 0, 0, implicit $exec :: (store (s64), addrspace 1)
; CHECK-NEXT: [[V_CNDMASK_B32_e64_1:%[0-9]+]]:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, $sgpr12_sgpr13, implicit $exec
; CHECK-NEXT: dead renamable $sgpr40_sgpr41 = V_CMP_NE_U32_e64 1, [[V_CNDMASK_B32_e64_1]], implicit $exec
- ; CHECK-NEXT: renamable $sgpr70 = S_ADD_U32 renamable $sgpr8, 32, implicit-def dead $scc
+ ; CHECK-NEXT: renamable $sgpr82 = S_ADD_U32 renamable $sgpr8, 32, implicit-def dead $scc
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: SI_SPILL_S64_SAVE killed renamable $sgpr4_sgpr5, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s64) into %stack.0, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr36_sgpr37 = lr-split COPY killed renamable $sgpr12_sgpr13
- ; CHECK-NEXT: renamable $sgpr38_sgpr39 = lr-split COPY killed renamable $sgpr14_sgpr15
+ ; CHECK-NEXT: renamable $sgpr38_sgpr39 = lr-split COPY killed renamable $sgpr74_sgpr75
; CHECK-NEXT: renamable $sgpr48_sgpr49 = lr-split COPY killed renamable $sgpr72_sgpr73
; CHECK-NEXT: renamable $sgpr50_sgpr51 = lr-split COPY killed renamable $sgpr18_sgpr19
; CHECK-NEXT: renamable $sgpr52_sgpr53 = lr-split COPY killed renamable $sgpr20_sgpr21
@@ -181,22 +181,26 @@ body: |
; CHECK-NEXT: SI_SPILL_S64_SAVE killed renamable $sgpr10_sgpr11, %stack.2, implicit $exec, implicit $sgpr32 :: (store (s64) into %stack.2, align 4, addrspace 5)
; CHECK-NEXT: $sgpr10_sgpr11 = SI_SPILL_S64_RESTORE %stack.2, implicit $exec, implicit $sgpr32 :: (load (s64) from %stack.2, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr64_sgpr65 = lr-split COPY killed renamable $sgpr24_sgpr25
- ; CHECK-NEXT: $sgpr12 = SI_SPILL_S32_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.4, addrspace 5)
- ; CHECK-NEXT: $sgpr13 = SI_SPILL_S32_RESTORE %stack.3, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.3, addrspace 5)
+ ; CHECK-NEXT: $sgpr12 = COPY renamable $sgpr14
+ ; CHECK-NEXT: $sgpr13 = COPY renamable $sgpr15
; CHECK-NEXT: renamable $sgpr66 = lr-split COPY killed renamable $sgpr8
; CHECK-NEXT: renamable $sgpr67 = lr-split COPY killed renamable $sgpr16
+ ; CHECK-NEXT: renamable $sgpr70 = lr-split COPY killed renamable $sgpr15
+ ; CHECK-NEXT: renamable $sgpr80 = lr-split COPY killed renamable $sgpr14
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr40_sgpr41, 0, csr_amdgpu_gfx90ainsts, implicit $sgpr4_sgpr5, implicit $sgpr6_sgpr7, implicit $sgpr10_sgpr11, implicit $sgpr12, implicit $sgpr13
; CHECK-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
- ; CHECK-NEXT: $sgpr8_sgpr9 = COPY renamable $sgpr70_sgpr71
+ ; CHECK-NEXT: $sgpr8_sgpr9 = COPY renamable $sgpr82_sgpr83
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr40_sgpr41, 0, csr_amdgpu_gfx90ainsts, implicit $sgpr8_sgpr9
+ ; CHECK-NEXT: renamable $sgpr14 = lr-split COPY killed renamable $sgpr80
+ ; CHECK-NEXT: renamable $sgpr15 = lr-split COPY killed renamable $sgpr70
; CHECK-NEXT: renamable $sgpr16 = lr-split COPY killed renamable $sgpr67
; CHECK-NEXT: renamable $sgpr24_sgpr25 = lr-split COPY killed renamable $sgpr64_sgpr65
; CHECK-NEXT: renamable $sgpr22_sgpr23 = lr-split COPY killed renamable $sgpr54_sgpr55
; CHECK-NEXT: renamable $sgpr20_sgpr21 = lr-split COPY killed renamable $sgpr52_sgpr53
; CHECK-NEXT: renamable $sgpr18_sgpr19 = lr-split COPY killed renamable $sgpr50_sgpr51
; CHECK-NEXT: renamable $sgpr72_sgpr73 = lr-split COPY killed renamable $sgpr48_sgpr49
- ; CHECK-NEXT: renamable $sgpr14_sgpr15 = lr-split COPY killed renamable $sgpr38_sgpr39
+ ; CHECK-NEXT: renamable $sgpr74_sgpr75 = lr-split COPY killed renamable $sgpr38_sgpr39
; CHECK-NEXT: renamable $sgpr12_sgpr13 = lr-split COPY killed renamable $sgpr36_sgpr37
; CHECK-NEXT: renamable $sgpr4_sgpr5 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr32 :: (load (s64) from %stack.0, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr6_sgpr7 = SI_SPILL_S64_RESTORE %stack.1, implicit $exec, implicit $sgpr32 :: (load (s64) from %stack.1, align 4, addrspace 5)
@@ -209,28 +213,28 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.10:
; CHECK-NEXT: successors: %bb.8(0x40000000), %bb.12(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.8, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.12
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.11:
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.1, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.17
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.12:
; CHECK-NEXT: successors: %bb.11(0x40000000), %bb.13(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $exec = S_MOV_B64_term killed renamable $sgpr24_sgpr25
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.11, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.13:
; CHECK-NEXT: successors: %bb.15(0x40000000), %bb.14(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr20_sgpr21, implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_VCCNZ %bb.15, implicit $vcc
@@ -238,18 +242,18 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.14:
; CHECK-NEXT: successors: %bb.15(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.15:
; CHECK-NEXT: successors: %bb.11(0x40000000), %bb.16(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr14_sgpr15, implicit-def dead $scc
+ ; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr74_sgpr75, implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_VCCNZ %bb.11, implicit $vcc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.16:
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
; CHECK-NEXT: {{ $}}
diff --git a/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir b/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir
index fe0dfd2eba37e9..b831d2eac61653 100644
--- a/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir
+++ b/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir
@@ -29,5 +29,5 @@ body: |
bb.2:
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
...
diff --git a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir
index 9b7411d98dc752..cc80d129acb461 100644
--- a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir
+++ b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir
@@ -4,12 +4,13 @@
# InlineSpiller tries to hoist the spill of %1 by storing its sibling %0 after
# the basic block prologue. Extending the PHI-defined %0 to that insertion point
# would overlap an interval already assigned to the same physical register.
-# Reject the hoist and spill %0 at the original copy instead.
+# Reject the hoist and spill %0 at the original copy instead. %3 is a COPY so it
+# is not rematerialized out of the prolog.
# CHECK-LABEL: name: hoist_spill_live_range_interference
# CHECK: bb.2:
# CHECK: SI_SPILL_WWM_V32_SAVE %0
-# CHECK-NEXT: $exec = S_AND_B64
+# CHECK-NEXT: $exec = bb-prolog S_AND_B64
---
name: hoist_spill_live_range_interference
@@ -35,10 +36,10 @@ body: |
%0:vgpr_32 = COPY %2
bb.2:
- %1:vgpr_32 = lr-split WWM_COPY %0:vgpr_32
- $exec = S_AND_B64 $exec, $exec, implicit-def $scc
- %3:vgpr_32 = IMPLICIT_DEF
- %4:vgpr_32 = IMPLICIT_DEF
+ %1:vgpr_32 = lr-split bb-prolog WWM_COPY %0:vgpr_32
+ $exec = bb-prolog S_AND_B64 $exec, $exec, implicit-def $scc
+ %3:vgpr_32 = bb-prolog COPY %2
+ %4:vgpr_32 = bb-prolog IMPLICIT_DEF
KILL %2:vgpr_32, %3:vgpr_32, %4:vgpr_32
KILL %1:vgpr_32, %2:vgpr_32
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir
index 31b2e4820aca14..71b92415491757 100644
--- a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir
+++ b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir
@@ -62,8 +62,8 @@ body: |
bb.2:
liveins: $sgpr0_sgpr1
- %1:vgpr_32 = lr-split WWM_COPY %0:vgpr_32
- $exec = S_AND_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ %1:vgpr_32 = lr-split bb-prolog WWM_COPY %0:vgpr_32
+ $exec = bb-prolog S_AND_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
KILL implicit-def $vgpr0, implicit-def $vgpr1, implicit-def $vgpr2, implicit-def $vgpr3, implicit-def $vgpr4, implicit-def $vgpr5, implicit-def $vgpr6, implicit-def $vgpr7, implicit-def $vgpr8, implicit-def $vgpr9, implicit-def $vgpr10, implicit-def $vgpr11, implicit-def $vgpr12, implicit-def $vgpr13, implicit-def $vgpr14, implicit-def $vgpr15, implicit-def $vgpr16, implicit-def $vgpr17, implicit-def $vgpr18, implicit-def $vgpr19, implicit-def $vgpr20, implicit-def $vgpr21, implicit-def $vgpr22, implicit-def $vgpr23, implicit-def $vgpr24, implicit-def $vgpr25, implicit-def $vgpr26, implicit-def $vgpr27, implicit-def $vgpr28, implicit-def $vgpr29, implicit-def $vgpr30, implicit-def $vgpr31, implicit-def $vgpr32, implicit-def $vgpr33, implicit-def $vgpr34, implicit-def $vgpr35, implicit-def $vgpr36, implicit-def $vgpr37, implicit-def $vgpr38, implicit-def $vgpr39, implicit-def $vgpr40, implicit-def $vgpr41, implicit-def $vgpr42, implicit-def $vgpr43, implicit-def $vgpr44, implicit-def $vgpr45, implicit-def $vgpr46, implicit-def $vgpr47, implicit-def $vgpr48, implicit-def $vgpr49, implicit-def $vgpr50, implicit-def $vgpr51, implicit-def $vgpr52, implicit-def $vgpr53, implicit-def $vgpr54, implicit-def $vgpr55, implicit-def $vgpr56, implicit-def $vgpr57, implicit-def $vgpr58, implicit-def $vgpr59, implicit-def $vgpr60, implicit-def $vgpr61, implicit-def $vgpr62, implicit-def $vgpr63, implicit-def $vgpr64, implicit-def $vgpr65, implicit-def $vgpr66, implicit-def $vgpr67, implicit-def $vgpr68, implicit-def $vgpr69, implicit-def $vgpr70, implicit-def $vgpr71, implicit-def $vgpr72, implicit-def $vgpr73, implicit-def $vgpr74, implicit-def $vgpr75, implicit-def $vgpr76, implicit-def $vgpr77, implicit-def $vgpr78, implicit-def $vgpr79, implicit-def $vgpr80, implicit-def $vgpr81, implicit-def $vgpr82, implicit-def $vgpr83, implicit-def $vgpr84, implicit-def $vgpr85, implicit-def $vgpr86, implicit-def $vgpr87, implicit-def $vgpr88, implicit-def $vgpr89, implicit-def $vgpr90, implicit-def $vgpr91, implicit-def $vgpr92, implicit-def $vgpr93, implicit-def $vgpr94, implicit-def $vgpr95, implicit-def $vgpr96, implicit-def $vgpr97, implicit-def $vgpr98, implicit-def $vgpr99, implicit-def $vgpr100, implicit-def $vgpr101, implicit-def $vgpr102, implicit-def $vgpr103, implicit-def $vgpr104, implicit-def $vgpr105, implicit-def $vgpr106, implicit-def $vgpr107, implicit-def $vgpr108, implicit-def $vgpr109, implicit-def $vgpr110, implicit-def $vgpr111, implicit-def $vgpr112, implicit-def $vgpr113, implicit-def $vgpr114, implicit-def $vgpr115, implicit-def $vgpr116, implicit-def $vgpr117, implicit-def $vgpr118, implicit-def $vgpr119, implicit-def $vgpr120, implicit-def $vgpr121, implicit-def $vgpr122, implicit-def $vgpr123, implicit-def $vgpr124, implicit-def $vgpr125, implicit-def $vgpr126, implicit-def $vgpr127, implicit-def $vgpr128, implicit-def $vgpr129, implicit-def $vgpr130, implicit-def $vgpr131, implicit-def $vgpr132, implicit-def $vgpr133, implicit-def $vgpr134, implicit-def $vgpr135, implicit-def $vgpr136, implicit-def $vgpr137, implicit-def $vgpr138, implicit-def $vgpr139, implicit-def $vgpr140, implicit-def $vgpr141, implicit-def $vgpr142, implicit-def $vgpr143, implicit-def $vgpr144, implicit-def $vgpr145, implicit-def $vgpr146, implicit-def $vgpr147, implicit-def $vgpr148, implicit-def $vgpr149, implicit-def $vgpr150, implicit-def $vgpr151, implicit-def $vgpr152, implicit-def $vgpr153, implicit-def $vgpr154, implicit-def $vgpr155, implicit-def $vgpr156, implicit-def $vgpr157, implicit-def $vgpr158, implicit-def $vgpr159, implicit-def $vgpr160, implicit-def $vgpr161, implicit-def $vgpr162, implicit-def $vgpr163, implicit-def $vgpr164, implicit-def $vgpr165, implicit-def $vgpr166, implicit-def $vgpr167, implicit-def $vgpr168, implicit-def $vgpr169, implicit-def $vgpr170, implicit-def $vgpr171, implicit-def $vgpr172, implicit-def $vgpr173, implicit-def $vgpr174, implicit-def $vgpr175, implicit-def $vgpr176, implicit-def $vgpr177, implicit-def $vgpr178, implicit-def $vgpr179, implicit-def $vgpr180, implicit-def $vgpr181, implicit-def $vgpr182, implicit-def $vgpr183, implicit-def $vgpr184, implicit-def $vgpr185, implicit-def $vgpr186, implicit-def $vgpr187, implicit-def $vgpr188, implicit-def $vgpr189, implicit-def $vgpr190, implicit-def $vgpr191, implicit-def $vgpr192, implicit-def $vgpr193, implicit-def $vgpr194, implicit-def $vgpr195, implicit-def $vgpr196, implicit-def $vgpr197, implicit-def $vgpr198, implicit-def $vgpr199, implicit-def $vgpr200, implicit-def $vgpr201, implicit-def $vgpr202, implicit-def $vgpr203, implicit-def $vgpr204, implicit-def $vgpr205, implicit-def $vgpr206, implicit-def $vgpr207, implicit-def $vgpr208, implicit-def $vgpr209, implicit-def $vgpr210, implicit-def $vgpr211, implicit-def $vgpr212, implicit-def $vgpr213, implicit-def $vgpr214, implicit-def $vgpr215, implicit-def $vgpr216, implicit-def $vgpr217, implicit-def $vgpr218, implicit-def $vgpr219, implicit-def $vgpr220, implicit-def $vgpr221, implicit-def $vgpr222, implicit-def $vgpr223, implicit-def $vgpr224, implicit-def $vgpr225, implicit-def $vgpr226, implicit-def $vgpr227, implicit-def $vgpr228, implicit-def $vgpr229, implicit-def $vgpr230, implicit-def $vgpr231, implicit-def $vgpr232, implicit-def $vgpr233, implicit-def $vgpr234, implicit-def $vgpr235, implicit-def $vgpr236, implicit-def $vgpr237, implicit-def $vgpr238, implicit-def $vgpr239, implicit-def $vgpr240, implicit-def $vgpr241, implicit-def $vgpr242, implicit-def $vgpr243, implicit-def $vgpr244, implicit-def $vgpr245, implicit-def $vgpr246, implicit-def $vgpr247, implicit-def $vgpr248, implicit-def $vgpr249, implicit-def $vgpr250, implicit-def $vgpr251, implicit-def $vgpr252, implicit-def $vgpr253, implicit-def $vgpr254, implicit-def $vgpr255
KILL %1:vgpr_32, %2:vgpr_32
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/rem_i128.ll b/llvm/test/CodeGen/AMDGPU/rem_i128.ll
index a52d709ca28cc4..72e94b6bbfc16d 100644
--- a/llvm/test/CodeGen/AMDGPU/rem_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/rem_i128.ll
@@ -679,12 +679,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB0_4
; GFX9-O0-NEXT: .LBB0_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
@@ -709,8 +703,13 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -737,7 +736,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -747,7 +745,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -756,11 +753,9 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -895,9 +890,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB0_6
; GFX9-O0-NEXT: s_branch .LBB0_1
; GFX9-O0-NEXT: .LBB0_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:332 ; 4-byte Folded Reload
@@ -914,9 +906,12 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -990,6 +985,7 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -1021,9 +1017,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB0_6
; GFX9-O0-NEXT: .LBB0_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
@@ -1032,14 +1025,17 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -1149,6 +1145,7 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -2018,12 +2015,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB1_4
; GFX9-O0-NEXT: .LBB1_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
@@ -2048,8 +2039,13 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -2076,7 +2072,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -2086,7 +2081,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -2095,11 +2089,9 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -2234,9 +2226,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB1_6
; GFX9-O0-NEXT: s_branch .LBB1_1
; GFX9-O0-NEXT: .LBB1_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
@@ -2253,9 +2242,12 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -2329,6 +2321,7 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -2360,9 +2353,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB1_6
; GFX9-O0-NEXT: .LBB1_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
@@ -2371,14 +2361,17 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -2488,6 +2481,7 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
diff --git a/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir b/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir
index 9a2a8b39f3c02d..846c24d9254edf 100644
--- a/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir
+++ b/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir
@@ -27,7 +27,7 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.3(0x40000000)
; CHECK-NEXT: liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.4, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
@@ -41,12 +41,12 @@ body: |
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.5:
; CHECK-NEXT: liveins: $vgpr1, $sgpr0_sgpr1
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_CVT_F32_U32_e32 killed $vgpr1, implicit $mode, implicit $exec
; CHECK-NEXT: SI_RETURN_TO_EPILOG killed $vgpr0
bb.0:
@@ -66,7 +66,7 @@ body: |
bb.2:
liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5
- $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
bb.3:
@@ -78,12 +78,12 @@ body: |
bb.4:
liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3
- $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
bb.5:
liveins: $vgpr1, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
renamable $vgpr0 = V_CVT_F32_U32_e32 killed $vgpr1, implicit $mode, implicit $exec
SI_RETURN_TO_EPILOG killed $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir b/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir
index f8fa2bda054c3f..ad50d3d9d35d87 100644
--- a/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir
+++ b/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir
@@ -18,8 +18,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
...
---
@@ -32,8 +32,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0_sgpr1
- renamable $sgpr0_sgpr1 = S_OR_SAVEEXEC_B64 killed renamable $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_XOR_B64 $exec, renamable $sgpr0_sgpr1, implicit-def $scc
+ renamable $sgpr0_sgpr1 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr0_sgpr1, implicit-def $scc
...
---
@@ -46,8 +46,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 renamable $sgpr0, $exec_lo, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 renamable $sgpr0, $exec_lo, implicit-def $scc
...
---
@@ -60,8 +60,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
...
---
@@ -75,8 +75,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr1, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 $exec_lo, renamable $sgpr1, implicit-def $scc
...
---
@@ -92,7 +92,7 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
$exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr1, implicit-def $scc
...
@@ -108,7 +108,7 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
...
---
diff --git a/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll b/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll
index 2fa366417ee73d..f6a9c48aaa3264 100644
--- a/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll
+++ b/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll
@@ -121,6 +121,9 @@ define amdgpu_kernel void @partial_no_vgprs_last_sgpr_spill(ptr addrspace(1) %ou
; GCN-NEXT: s_cbranch_vccnz .LBB0_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[24:25], -1
+; GCN-NEXT: buffer_load_dword v22, off, s[0:3], 0 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[24:25]
+; GCN-NEXT: s_or_saveexec_b64 s[24:25], -1
; GCN-NEXT: buffer_load_dword v23, off, s[0:3], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[24:25]
; GCN-NEXT: s_waitcnt vmcnt(0)
@@ -140,9 +143,6 @@ define amdgpu_kernel void @partial_no_vgprs_last_sgpr_spill(ptr addrspace(1) %ou
; GCN-NEXT: v_readlane_b32 s17, v23, 13
; GCN-NEXT: v_readlane_b32 s18, v23, 14
; GCN-NEXT: v_readlane_b32 s19, v23, 15
-; GCN-NEXT: s_or_saveexec_b64 s[24:25], -1
-; GCN-NEXT: buffer_load_dword v22, off, s[0:3], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[24:25]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[4:19]
; GCN-NEXT: ;;#ASMEND
@@ -200,7 +200,6 @@ define amdgpu_kernel void @partial_no_vgprs_last_sgpr_spill(ptr addrspace(1) %ou
; GCN-NEXT: v_readlane_b32 s21, v23, 61
; GCN-NEXT: v_readlane_b32 s22, v23, 62
; GCN-NEXT: v_readlane_b32 s23, v23, 63
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s4, v22, 0
; GCN-NEXT: v_readlane_b32 s5, v22, 1
; GCN-NEXT: ;;#ASMSTART
diff --git a/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir b/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir
index 55ec919041e001..bc20ccf85df9a4 100644
--- a/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir
+++ b/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir
@@ -40,145 +40,145 @@ body: |
; GCN-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr5, $sgpr33, 16, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" store (s32) into %stack.72, addrspace 5)
; GCN-NEXT: $exec_lo = S_MOV_B32 killed $sgpr0
; GCN-NEXT: $sgpr32 = frame-setup S_ADD_I32 $sgpr32, 24, implicit-def dead $scc
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr4, 0, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr4, 0, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr4, $vgpr2, 0, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr5, 1, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr5, 1, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr5, $vgpr2, 1, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr6, 2, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr6, 2, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr6, $vgpr2, 2, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr7, 3, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr7, 3, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr7, $vgpr2, 3, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr8, 4, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr8, 4, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr8, $vgpr2, 4, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr9, 5, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr9, 5, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr9, $vgpr2, 5, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr10, 6, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr10, 6, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr10, $vgpr2, 6, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr11, 7, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr11, 7, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr11, $vgpr2, 7, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr12, 8, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr12, 8, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr12, $vgpr2, 8, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr13, 9, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr13, 9, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr13, $vgpr2, 9, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr14, 10, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr14, 10, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr14, $vgpr2, 10, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr15, 11, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr15, 11, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr15, $vgpr2, 11, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr16, 12, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr16, 12, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr16, $vgpr2, 12, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr17, 13, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr17, 13, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr17, $vgpr2, 13, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr18, 14, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr18, 14, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr18, $vgpr2, 14, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr19, 15, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr19, 15, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr19, $vgpr2, 15, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr20, 16, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr20, 16, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr20, $vgpr2, 16, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr21, 17, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr21, 17, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr21, $vgpr2, 17, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr22, 18, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr22, 18, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr22, $vgpr2, 18, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr23, 19, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr23, 19, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr23, $vgpr2, 19, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr24, 20, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr24, 20, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr24, $vgpr2, 20, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr25, 21, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr25, 21, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr25, $vgpr2, 21, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr26, 22, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr26, 22, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr26, $vgpr2, 22, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr27, 23, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr27, 23, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr27, $vgpr2, 23, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr28, 24, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr28, 24, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr28, $vgpr2, 24, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr29, 25, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr29, 25, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr29, $vgpr2, 25, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr64, 26, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr64, 26, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr64, $vgpr2, 26, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr65, 27, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr65, 27, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr65, $vgpr2, 27, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr66, 28, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr66, 28, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr66, $vgpr2, 28, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr67, 29, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr67, 29, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr67, $vgpr2, 29, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr68, 30, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr68, 30, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr68, $vgpr2, 30, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr69, 31, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr69, 31, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr69, $vgpr2, 31, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr70, 0, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr70, 0, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr70, $vgpr3, 0, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr71, 1, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr71, 1, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr71, $vgpr3, 1, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr72, 2, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr72, 2, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr72, $vgpr3, 2, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr73, 3, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr73, 3, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr73, $vgpr3, 3, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr74, 4, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr74, 4, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr74, $vgpr3, 4, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr75, 5, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr75, 5, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr75, $vgpr3, 5, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr76, 6, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr76, 6, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr76, $vgpr3, 6, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr77, 7, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr77, 7, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr77, $vgpr3, 7, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr78, 8, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr78, 8, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr78, $vgpr3, 8, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr79, 9, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr79, 9, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr79, $vgpr3, 9, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr80, 10, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr80, 10, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr80, $vgpr3, 10, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr81, 11, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr81, 11, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr81, $vgpr3, 11, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr82, 12, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr82, 12, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr82, $vgpr3, 12, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr83, 13, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr83, 13, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr83, $vgpr3, 13, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr84, 14, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr84, 14, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr84, $vgpr3, 14, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr85, 15, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr85, 15, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr85, $vgpr3, 15, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr86, 16, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr86, 16, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr86, $vgpr3, 16, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr87, 17, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr87, 17, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr87, $vgpr3, 17, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr88, 18, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr88, 18, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr88, $vgpr3, 18, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr89, 19, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr89, 19, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr89, $vgpr3, 19, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr90, 20, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr90, 20, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr90, $vgpr3, 20, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr91, 21, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr91, 21, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr91, $vgpr3, 21, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr92, 22, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr92, 22, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr92, $vgpr3, 22, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr93, 23, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr93, 23, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr93, $vgpr3, 23, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr94, 24, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr94, 24, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr94, $vgpr3, 24, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr95, 25, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr95, 25, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr95, $vgpr3, 25, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr96, 26, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr96, 26, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr96, $vgpr3, 26, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr97, 27, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr97, 27, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr97, $vgpr3, 27, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr98, 28, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr98, 28, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr98, $vgpr3, 28, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr99, 29, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr99, 29, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr99, $vgpr3, 29, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr100, 30, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr100, 30, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr100, $vgpr3, 30, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr101, 31, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr101, 31, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr101, $vgpr3, 31, 32
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr102, 0, $vgpr4
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr102, 0, $vgpr4
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr102, $vgpr4, 0, 32
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr103, 1, $vgpr4
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr103, 1, $vgpr4
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr103, $vgpr4, 1, 32
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr30, 2, $vgpr4, implicit-def $sgpr30_sgpr31, implicit $sgpr30_sgpr31
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr31, 3, $vgpr4, implicit $sgpr30_sgpr31
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr30, 2, $vgpr4, implicit-def $sgpr30_sgpr31, implicit $sgpr30_sgpr31
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr31, 3, $vgpr4, implicit $sgpr30_sgpr31
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $pc_reg, $vgpr4, 2, 32, $vgpr4, 3, 32
- ; GCN-NEXT: $sgpr22 = IMPLICIT_DEF
- ; GCN-NEXT: $vgpr5 = IMPLICIT_DEF
- ; GCN-NEXT: $vgpr5 = SI_SPILL_S32_TO_VGPR $sgpr22, 0, killed $vgpr5
- ; GCN-NEXT: $sgpr0 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: $sgpr22 = bb-prolog IMPLICIT_DEF
+ ; GCN-NEXT: $vgpr5 = bb-prolog IMPLICIT_DEF
+ ; GCN-NEXT: $vgpr5 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr22, 0, killed $vgpr5
+ ; GCN-NEXT: $sgpr0 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: dead $vgpr1 = V_SET_INACTIVE_B32 0, $vgpr0, 0, 0, $sgpr_null, implicit $exec, implicit-def $scc
; GCN-NEXT: $exec_lo = EXIT_STRICT_WWM $sgpr0
; GCN-NEXT: {{ $}}
@@ -278,9 +278,9 @@ body: |
bb.0:
liveins: $vgpr0
- $sgpr22 = IMPLICIT_DEF
- SI_SPILL_S32_SAVE $sgpr22, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.0, addrspace 5)
- $sgpr0 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr22 = bb-prolog IMPLICIT_DEF
+ bb-prolog SI_SPILL_S32_SAVE $sgpr22, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.0, addrspace 5)
+ $sgpr0 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
%0:vgpr_32 = V_SET_INACTIVE_B32 0, $vgpr0, 0, 0, $sgpr_null, implicit $exec, implicit-def $scc
$exec_lo = EXIT_STRICT_WWM $sgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir b/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir
index 67b6f10417ea42..0111f21e64c314 100644
--- a/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir
@@ -31,7 +31,7 @@ body: |
S_BRANCH %bb.1
bb.2:
- SI_END_CF %1, implicit-def $exec, implicit-def $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def $exec, implicit-def $scc, implicit $exec
%11 = S_MOV_B32 1
%2 = S_ADD_I32 %0, %11, implicit-def $scc
S_BRANCH %bb.1
diff --git a/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir b/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir
index 03d66f82ab8450..3aaf765a80e912 100644
--- a/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir
@@ -16,7 +16,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr8
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; CHECK-NEXT: undef [[COPY1:%[0-9]+]].sub0:ccr_sgpr_64 = COPY $sgpr1
; CHECK-NEXT: [[COPY1:%[0-9]+]].sub1:ccr_sgpr_64 = COPY $sgpr2
@@ -33,7 +33,7 @@ body: |
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = V_ADD_U32_e64 5, [[COPY2]], 0, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY3]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY3]], implicit-def $scc
; CHECK-NEXT: $vgpr8 = COPY [[COPY2]]
; CHECK-NEXT: $sgpr0 = COPY [[COPY]]
; CHECK-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY $sgpr0
@@ -56,7 +56,7 @@ body: |
%37:vgpr_32 = V_ADD_U32_e64 5, %37:vgpr_32, 0, implicit $exec
bb.2:
- $exec_lo = S_OR_B32 $exec_lo, %16:sreg_32_xm0_xexec, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %16:sreg_32_xm0_xexec, implicit-def $scc
$vgpr8 = COPY %37:vgpr_32
$sgpr0 = COPY %9:sreg_32
%2:sreg_32 = COPY $sgpr0
@@ -76,7 +76,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr8, $vgpr9
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; CHECK-NEXT: undef [[COPY1:%[0-9]+]].sub0:ccr_sgpr_64 = COPY $sgpr1
; CHECK-NEXT: [[COPY1:%[0-9]+]].sub1:ccr_sgpr_64 = COPY $sgpr2
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir
index e0312a4d3e3295..c82b84f09a18d7 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir
@@ -57,9 +57,9 @@ body: |
bb.3:
successors: %bb.4(0x80000000)
- SI_END_CF killed %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF killed %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir
index b69d1f51bb299e..e5c4b40027971c 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir
@@ -34,7 +34,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, [[COPY3]], implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY3]], implicit-def $scc
; CHECK-NEXT: [[REG_SEQUENCE:%[0-9]+]]:vreg_64 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY1]], %subreg.sub1
; CHECK-NEXT: S_BRANCH %bb.5
; CHECK-NEXT: {{ $}}
@@ -70,7 +70,7 @@ body: |
S_BRANCH %bb.3
bb.2:
- SI_END_CF killed %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%7:vreg_64 = REG_SEQUENCE killed %2, %subreg.sub0, killed %1, %subreg.sub1
S_BRANCH %bb.5
@@ -86,7 +86,7 @@ body: |
S_BRANCH %bb.4
bb.4:
- SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.2
bb.5:
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir
index 109cd0c5b61c49..e79a49ce36af93 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir
@@ -129,7 +129,7 @@ body: |
successors: %bb.4(0x04000000), %bb.1(0x7c000000)
%20:vreg_1 = PHI %26, %bb.2, %19, %bb.1 ;%20:vreg_1 = PHI %19, %bb.1, %26, %bb.2 - this is original phi created by SDAG
- SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%27:sreg_64 = S_MOV_B64 4
%18:vreg_64 = V_ADD_U64_PSEUDO %17, killed %27, implicit-def dead $vcc, implicit $exec
%28:sreg_32 = S_MOV_B32 1
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir
index aee14ba54125f9..5722441f4d752d 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir
@@ -57,11 +57,11 @@ body: |
bb.2:
%6:vreg_1 = PHI %5, %bb.1
- SI_END_CF %3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
%7:vreg_1 = PHI %6, %bb.2, %8, %bb.0
- SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir
index d000f5a30b6ddb..01a6e2d4f3750c 100644
--- a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir
@@ -94,7 +94,7 @@ body: |
bb.8:
%13:vreg_128 = PHI %10, %bb.6, %12, %bb.7
- SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%5:vreg_128 = REG_SEQUENCE %13.sub2, %subreg.sub0, %13.sub2, %subreg.sub1, killed %13.sub2, %subreg.sub2, undef %3, %subreg.sub3
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir
index 23484ddef17abc..1729aa9d212e3c 100644
--- a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir
@@ -76,6 +76,6 @@ body: |
bb.5:
%6:vreg_64 = PHI undef %7:vreg_64, %bb.3, %1, %bb.4
- SI_END_CF killed %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit %6
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir b/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir
index 9337d244c71a64..898d26252d9d53 100644
--- a/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir
@@ -106,7 +106,7 @@ body: |
S_BRANCH %bb.1
bb.4:
- SI_END_CF %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
DBG_VALUE_LIST !4, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_constu, 2712, DW_OP_mul, DW_OP_plus, DW_OP_plus_uconst, 2680, DW_OP_stack_value), %5, 0, debug-location !9
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir b/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir
index 14a94de24de228..64ef9f178fcb7c 100644
--- a/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir
@@ -23,8 +23,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr1
- %0:vgpr_32 = IMPLICIT_DEF
- renamable $sgpr4_sgpr5 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:vgpr_32 = bb-prolog IMPLICIT_DEF
+ renamable $sgpr4_sgpr5 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
%1:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
%2:vgpr_32 = V_MOV_B32_dpp %1, %0, 323, 12, 15, 0, implicit $exec
$exec = EXIT_STRICT_WWM killed renamable $sgpr4_sgpr5
diff --git a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir
index c0f0574411a6a6..2ed42e286ba815 100644
--- a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir
+++ b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir
@@ -28,7 +28,7 @@ body: |
; GFX10-NEXT: successors: %bb.2(0x40000000), %bb.3(0x40000000)
; GFX10-NEXT: liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.2
@@ -55,7 +55,7 @@ body: |
successors: %bb.2(0x40000000), %bb.8(0x40000000)
liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
- $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.8, implicit $exec
S_BRANCH %bb.2
@@ -96,7 +96,7 @@ body: |
; GFX10-NEXT: successors: %bb.2(0x40000000), %bb.3(0x40000000)
; GFX10-NEXT: liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.2
@@ -123,7 +123,7 @@ body: |
successors: %bb.2(0x40000000), %bb.8(0x40000000)
liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
- $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.8, implicit $exec
S_BRANCH %bb.2
@@ -164,8 +164,8 @@ body: |
; GFX10-NEXT: successors: %bb.2(0x40000000), %bb.3(0x40000000)
; GFX10-NEXT: liveins: $sgpr6, $sgpr8, $sgpr0_sgpr1, $sgpr2_sgpr3
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr2 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
- ; GFX10-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr2 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_NOP 0, implicit $sgpr2_sgpr3
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
@@ -193,8 +193,8 @@ body: |
successors: %bb.2(0x40000000), %bb.8(0x40000000)
liveins: $sgpr6, $sgpr8, $sgpr0_sgpr1, $sgpr2_sgpr3
- $sgpr2 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
- $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr2 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
S_NOP 0, implicit $sgpr2_sgpr3
S_CBRANCH_EXECZ %bb.8, implicit $exec
@@ -226,7 +226,7 @@ body: |
; GFX10-NEXT: successors: %bb.7(0x40000000), %bb.2(0x40000000)
; GFX10-NEXT: liveins: $sgpr8, $vgpr0
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.7, implicit $exec
; GFX10-NEXT: {{ $}}
@@ -238,7 +238,7 @@ body: |
; GFX10-NEXT: successors: %bb.5(0x40000000), %bb.4(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $vgpr0
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.4
@@ -262,7 +262,7 @@ body: |
; GFX10-NEXT: successors: %bb.3(0x80000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr5, $sgpr8, $vgpr0, $vgpr1
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
; GFX10-NEXT: INLINEASM &"", sideeffect attdialect, implicit killed renamable $vgpr1
; GFX10-NEXT: S_BRANCH %bb.3
; GFX10-NEXT: {{ $}}
@@ -270,7 +270,7 @@ body: |
; GFX10-NEXT: successors: %bb.1(0x40000000), %bb.8(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $vgpr0
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
; GFX10-NEXT: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: bb.8:
@@ -290,7 +290,7 @@ body: |
bb.1:
liveins: $sgpr8, $vgpr0
- $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -301,7 +301,7 @@ body: |
bb.2:
liveins: $sgpr4, $sgpr8, $vgpr0
- $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
@@ -324,14 +324,14 @@ body: |
bb.5:
liveins: $sgpr4, $sgpr5, $sgpr8, $vgpr0, $vgpr1
- $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
INLINEASM &"", sideeffect attdialect, implicit killed renamable $vgpr1
S_BRANCH %bb.2
bb.6:
liveins: $sgpr4, $sgpr8, $vgpr0
- $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.10:
@@ -358,7 +358,7 @@ body: |
; GFX10-NEXT: successors: %bb.7(0x40000000), %bb.2(0x40000000)
; GFX10-NEXT: liveins: $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.7, implicit $exec
; GFX10-NEXT: {{ $}}
@@ -370,7 +370,7 @@ body: |
; GFX10-NEXT: successors: %bb.5(0x40000000), %bb.4(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.4
@@ -395,7 +395,7 @@ body: |
; GFX10-NEXT: successors: %bb.3(0x80000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr5, $sgpr6, $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
; GFX10-NEXT: INLINEASM &"", sideeffect attdialect, implicit killed renamable $sgpr6
; GFX10-NEXT: S_BRANCH %bb.3
; GFX10-NEXT: {{ $}}
@@ -403,7 +403,7 @@ body: |
; GFX10-NEXT: successors: %bb.1(0x40000000), %bb.8(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
; GFX10-NEXT: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: bb.8:
@@ -424,7 +424,7 @@ body: |
bb.1:
liveins: $sgpr8, $sgpr9
- $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -435,7 +435,7 @@ body: |
bb.2:
liveins: $sgpr4, $sgpr8, $sgpr9
- $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
@@ -458,14 +458,14 @@ body: |
bb.5:
liveins: $sgpr4, $sgpr5, $sgpr6, $sgpr8, $sgpr9
- $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
INLINEASM &"", sideeffect attdialect, implicit killed renamable $sgpr6
S_BRANCH %bb.2
bb.6:
liveins: $sgpr4, $sgpr8, $sgpr9
- $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.10:
diff --git a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir
index 653c2c6f5475be..95030c8650acdc 100644
--- a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir
+++ b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir
@@ -36,7 +36,7 @@ body: |
; GFX10-NEXT: bb.2:
; GFX10-NEXT: successors: %bb.3(0x40000000), %bb.4(0x40000000)
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_XOR_B32_1]], implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_XOR_B32_1]], implicit-def $scc
; GFX10-NEXT: [[V_LSHRREV_B32_e64_:%[0-9]+]]:vgpr_32 = V_LSHRREV_B32_e64 [[S_MOV_B32_]], [[DEF]], implicit $exec
; GFX10-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sreg_32 = S_MOV_B32 31
; GFX10-NEXT: [[V_CMP_NE_U32_e64_1:%[0-9]+]]:sreg_32 = V_CMP_NE_U32_e64 [[V_BFE_U32_e64_]], killed [[S_MOV_B32_2]], implicit $exec
@@ -55,7 +55,7 @@ body: |
; GFX10-NEXT: bb.4:
; GFX10-NEXT: successors: %bb.5(0x80000000)
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_XOR_B32_3]], implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_XOR_B32_3]], implicit-def $scc
; GFX10-NEXT: [[DEF2:%[0-9]+]]:sreg_32 = IMPLICIT_DEF
; GFX10-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sreg_32 = S_MOV_B32 16
; GFX10-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sreg_32 = S_MOV_B32 4
@@ -63,7 +63,7 @@ body: |
; GFX10-NEXT: S_BRANCH %bb.5
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: bb.5:
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[DEF2]], implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[DEF2]], implicit-def $scc
; GFX10-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.4(0x40000000), %bb.5(0x40000000)
@@ -91,7 +91,7 @@ body: |
bb.5:
successors: %bb.6(0x40000000), %bb.7(0x40000000)
- $exec_lo = S_OR_B32 $exec_lo, %23, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %23, implicit-def $scc
%24:sreg_32 = S_MOV_B32 31
%25:sreg_32 = V_CMP_NE_U32_e64 %18, killed %24, implicit $exec
%26:sreg_32 = S_XOR_B32 %25, -1, implicit-def $scc
@@ -109,14 +109,14 @@ body: |
bb.7:
successors: %bb.8(0x80000000)
- $exec_lo = S_OR_B32 $exec_lo, %28, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %28, implicit-def $scc
%29:sreg_32 = S_MOV_B32 16
%30:sreg_32 = S_MOV_B32 4
%31:vgpr_32 = nuw nsw V_LSHL_ADD_U32_e64 %17, %30, killed %29, implicit $exec
S_BRANCH %bb.8
bb.8:
- $exec_lo = S_OR_B32 $exec_lo, %14, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %14, implicit-def $scc
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir b/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir
index a4b34780879ef7..616b3c8ca88026 100644
--- a/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir
+++ b/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir
@@ -30,15 +30,15 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr0_sgpr1, $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
- ; CHECK-NEXT: $exec = S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.2, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: liveins: $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
; CHECK-NEXT: S_ENDPGM 0
; CHECK-NEXT: {{ $}}
@@ -86,15 +86,15 @@ body: |
successors: %bb.3, %bb.4
liveins: $sgpr0_sgpr1, $sgpr2_sgpr3
- renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
+ renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
bb.4:
liveins: $sgpr2_sgpr3
- $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
S_ENDPGM 0
@@ -141,15 +141,15 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr6_sgpr7, $sgpr30_sgpr31, $vgpr1_vgpr2
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr6_sgpr7 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
- ; CHECK-NEXT: $exec = S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
+ ; CHECK-NEXT: renamable $sgpr6_sgpr7 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.2, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: liveins: $sgpr6_sgpr7, $sgpr30_sgpr31
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
; CHECK-NEXT: S_SETPC_B64_return $sgpr30_sgpr31
; CHECK-NEXT: {{ $}}
@@ -185,15 +185,15 @@ body: |
successors: %bb.3, %bb.4
liveins: $sgpr6_sgpr7, $sgpr30_sgpr31, $vgpr1_vgpr2
- renamable $sgpr6_sgpr7 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
+ renamable $sgpr6_sgpr7 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
bb.4:
liveins: $sgpr6_sgpr7, $sgpr30_sgpr31
- $exec = S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
S_SETPC_B64_return $sgpr30_sgpr31
diff --git a/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir b/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir
index 27a0f9ea3a6a44..cf03f0be4a20cf 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir
+++ b/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir
@@ -47,7 +47,7 @@ body: |
bb.1:
liveins: $sgpr96_sgpr97, $sgpr98_sgpr99, $sgpr102_sgpr103
- %0:sreg_64 = S_OR_SAVEEXEC_B64 $sgpr96_sgpr97, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr96_sgpr97, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %0, implicit-def $scc
S_CBRANCH_EXECZ %bb.3, implicit $exec
S_BRANCH %bb.2
@@ -55,7 +55,7 @@ body: |
bb.2:
liveins: $sgpr98_sgpr99, $sgpr102_sgpr103
- %0:sreg_64 = S_OR_SAVEEXEC_B64 $sgpr98_sgpr99, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr98_sgpr99, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %0, implicit-def $scc
S_CBRANCH_EXECZ %bb.3, implicit $exec
S_BRANCH %bb.4
@@ -63,7 +63,7 @@ body: |
bb.3:
liveins: $sgpr102_sgpr103
- %0:sreg_64 = S_OR_SAVEEXEC_B64 $sgpr102_sgpr103, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr102_sgpr103, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %0, implicit-def $scc
S_BRANCH %bb.4
diff --git a/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll b/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll
index d45833b1fc262b..9ba2cbb2f29dc2 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll
@@ -9741,36 +9741,23 @@ entry:
define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspace(1) %in) #0 {
; GFX6-LABEL: test_limited_sgpr:
; GFX6: ; %bb.0: ; %entry
-; GFX6-NEXT: s_mov_b32 s40, SCRATCH_RSRC_DWORD0
-; GFX6-NEXT: s_mov_b32 s41, SCRATCH_RSRC_DWORD1
; GFX6-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x9
-; GFX6-NEXT: s_mov_b32 s42, -1
-; GFX6-NEXT: s_mov_b32 s43, 0xe8f000
-; GFX6-NEXT: s_add_u32 s40, s40, s11
; GFX6-NEXT: v_mbcnt_lo_u32_b32_e64 v0, -1, 0
-; GFX6-NEXT: s_addc_u32 s41, s41, 0
; GFX6-NEXT: v_mbcnt_hi_u32_b32_e32 v0, -1, v0
-; GFX6-NEXT: v_mov_b32_e32 v6, 0
-; GFX6-NEXT: s_mov_b64 s[4:5], exec
; GFX6-NEXT: s_mov_b32 s6, 0
+; GFX6-NEXT: v_mov_b32_e32 v6, 0
; GFX6-NEXT: s_mov_b32 s7, 0xf000
-; GFX6-NEXT: s_mov_b64 exec, 15
-; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], 0
-; GFX6-NEXT: s_waitcnt expcnt(0) lgkmcnt(0)
-; GFX6-NEXT: v_writelane_b32 v1, s0, 0
-; GFX6-NEXT: v_writelane_b32 v1, s1, 1
-; GFX6-NEXT: v_writelane_b32 v1, s2, 2
-; GFX6-NEXT: v_writelane_b32 v1, s3, 3
-; GFX6-NEXT: s_mov_b32 s8, 0x80400
-; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s8 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v1, off, s[40:43], 0
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[4:5]
+; GFX6-NEXT: s_waitcnt lgkmcnt(0)
; GFX6-NEXT: s_mov_b64 s[4:5], s[2:3]
; GFX6-NEXT: v_lshlrev_b32_e32 v5, 8, v0
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:240
-; GFX6-NEXT: s_mov_b32 s2, 0x86800
+; GFX6-NEXT: s_mov_b32 s40, SCRATCH_RSRC_DWORD0
+; GFX6-NEXT: s_mov_b32 s41, SCRATCH_RSRC_DWORD1
+; GFX6-NEXT: s_mov_b32 s42, -1
+; GFX6-NEXT: s_mov_b32 s43, 0xe8f000
+; GFX6-NEXT: s_add_u32 s40, s40, s11
+; GFX6-NEXT: s_addc_u32 s41, s41, 0
+; GFX6-NEXT: s_mov_b32 s2, 0x85c00
; GFX6-NEXT: s_mov_b64 s[8:9], exec
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
@@ -9779,7 +9766,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:224
-; GFX6-NEXT: s_mov_b32 s2, 0x86400
+; GFX6-NEXT: s_mov_b32 s2, 0x85800
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9787,7 +9774,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:208
-; GFX6-NEXT: s_mov_b32 s2, 0x86000
+; GFX6-NEXT: s_mov_b32 s2, 0x85400
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9795,7 +9782,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:192
-; GFX6-NEXT: s_mov_b32 s2, 0x85c00
+; GFX6-NEXT: s_mov_b32 s2, 0x85000
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9803,7 +9790,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:176
-; GFX6-NEXT: s_mov_b32 s2, 0x85800
+; GFX6-NEXT: s_mov_b32 s2, 0x84c00
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9811,7 +9798,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:160
-; GFX6-NEXT: s_mov_b32 s2, 0x85400
+; GFX6-NEXT: s_mov_b32 s2, 0x84800
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9819,7 +9806,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:144
-; GFX6-NEXT: s_mov_b32 s2, 0x85000
+; GFX6-NEXT: s_mov_b32 s2, 0x84400
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9827,7 +9814,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:128
-; GFX6-NEXT: s_mov_b32 s2, 0x84c00
+; GFX6-NEXT: s_mov_b32 s2, 0x84000
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9835,7 +9822,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:112
-; GFX6-NEXT: s_mov_b32 s2, 0x84800
+; GFX6-NEXT: s_mov_b32 s2, 0x83c00
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9843,7 +9830,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:96
-; GFX6-NEXT: s_mov_b32 s2, 0x84400
+; GFX6-NEXT: s_mov_b32 s2, 0x83800
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9851,20 +9838,14 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:80
-; GFX6-NEXT: s_mov_b32 s2, 0x84000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:64
-; GFX6-NEXT: s_mov_b32 s2, 0x83800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_load_dwordx4 v[12:15], v[5:6], s[4:7], 0 addr64 offset:32
+; GFX6-NEXT: s_mov_b32 s2, 0x83400
+; GFX6-NEXT: s_waitcnt vmcnt(1)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_load_dwordx4 v[16:19], v[5:6], s[4:7], 0 addr64 offset:64
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64
; GFX6-NEXT: buffer_load_dwordx4 v[7:10], v[5:6], s[4:7], 0 addr64 offset:16
@@ -9874,14 +9855,6 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v8, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v9, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v10, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dwordx4 v[7:10], v[5:6], s[4:7], 0 addr64 offset:32
-; GFX6-NEXT: s_mov_b32 s2, 0x83400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dword v7, off, s[40:43], s2 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v8, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v9, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v10, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_mov_b64 s[2:3], s[6:7]
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
@@ -9890,24 +9863,17 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s1, 1
; GFX6-NEXT: v_writelane_b32 v4, s2, 2
; GFX6-NEXT: v_writelane_b32 v4, s3, 3
-; GFX6-NEXT: s_mov_b32 s10, 0x80800
+; GFX6-NEXT: s_mov_b32 s10, 0x80400
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s10 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: s_mov_b64 exec, s[8:9]
-; GFX6-NEXT: buffer_load_dwordx4 v[7:10], v[5:6], s[4:7], 0 addr64 offset:48
-; GFX6-NEXT: s_mov_b32 s0, 0x83c00
+; GFX6-NEXT: buffer_load_dwordx4 v[20:23], v[5:6], s[4:7], 0 addr64 offset:48
; GFX6-NEXT: v_lshlrev_b32_e32 v4, 13, v0
; GFX6-NEXT: v_add_i32_e32 v4, vcc, 16, v4
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dword v7, off, s[40:43], s0 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(3)
; GFX6-NEXT: v_mov_b32_e32 v7, 1
-; GFX6-NEXT: s_mov_b64 s[0:1], exec
+; GFX6-NEXT: s_mov_b64 s[2:3], exec
; GFX6-NEXT: buffer_store_dword v7, v4, s[40:43], 0 offen
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; def s[4:11]
@@ -9923,12 +9889,12 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s9, 5
; GFX6-NEXT: v_writelane_b32 v4, s10, 6
; GFX6-NEXT: v_writelane_b32 v4, s11, 7
-; GFX6-NEXT: s_mov_b32 s2, 0x80c00
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s12, 0x80800
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[0:1]
+; GFX6-NEXT: s_mov_b64 exec, s[2:3]
; GFX6-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; def s[8:15]
@@ -9940,10 +9906,10 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: ; def s[24:31]
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
-; GFX6-NEXT: ; def s[0:3]
+; GFX6-NEXT: ; def s[4:7]
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
-; GFX6-NEXT: ; def s[4:5]
+; GFX6-NEXT: ; def s[2:3]
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; def s33
@@ -9951,7 +9917,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: s_and_saveexec_b64 s[34:35], vcc
; GFX6-NEXT: s_cbranch_execz .LBB1_2
; GFX6-NEXT: ; %bb.1: ; %bb0
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -9963,18 +9929,18 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s13, 5
; GFX6-NEXT: v_writelane_b32 v4, s14, 6
; GFX6-NEXT: v_writelane_b32 v4, s15, 7
-; GFX6-NEXT: s_mov_b32 s36, 0x81400
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s38, 0x81000
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s36, 0x80c00
+; GFX6-NEXT: s_mov_b32 s38, 0x80800
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s36 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: v_readlane_b32 s8, v4, 0
; GFX6-NEXT: v_readlane_b32 s9, v4, 1
@@ -9986,8 +9952,8 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_readlane_b32 s15, v4, 7
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -9999,18 +9965,18 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s21, 5
; GFX6-NEXT: v_writelane_b32 v4, s22, 6
; GFX6-NEXT: v_writelane_b32 v4, s23, 7
-; GFX6-NEXT: s_mov_b32 s36, 0x81c00
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s38, 0x81800
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s36, 0x81400
+; GFX6-NEXT: s_mov_b32 s38, 0x81000
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s36 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: v_readlane_b32 s16, v4, 0
; GFX6-NEXT: v_readlane_b32 s17, v4, 1
@@ -10022,8 +9988,8 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_readlane_b32 s23, v4, 7
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -10035,18 +10001,18 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s29, 5
; GFX6-NEXT: v_writelane_b32 v4, s30, 6
; GFX6-NEXT: v_writelane_b32 v4, s31, 7
-; GFX6-NEXT: s_mov_b32 s36, 0x82400
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s38, 0x82000
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s36, 0x81c00
+; GFX6-NEXT: s_mov_b32 s38, 0x81800
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s36 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: v_readlane_b32 s24, v4, 0
; GFX6-NEXT: v_readlane_b32 s25, v4, 1
@@ -10058,8 +10024,8 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_readlane_b32 s31, v4, 7
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -10067,17 +10033,31 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s1, 1
; GFX6-NEXT: v_writelane_b32 v4, s2, 2
; GFX6-NEXT: v_writelane_b32 v4, s3, 3
-; GFX6-NEXT: s_mov_b32 s36, 0x82c00
+; GFX6-NEXT: s_mov_b32 s38, 0x82c00
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[0:1], exec
+; GFX6-NEXT: s_mov_b64 exec, 15
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: v_writelane_b32 v4, s4, 0
+; GFX6-NEXT: v_writelane_b32 v4, s5, 1
+; GFX6-NEXT: v_writelane_b32 v4, s6, 2
+; GFX6-NEXT: v_writelane_b32 v4, s7, 3
+; GFX6-NEXT: s_mov_b32 s36, 0x82800
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 vcc, s[4:5]
+; GFX6-NEXT: s_mov_b64 exec, s[0:1]
+; GFX6-NEXT: s_mov_b64 vcc, s[2:3]
; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s38, 0x82400
+; GFX6-NEXT: s_mov_b32 s38, 0x82000
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
@@ -10095,7 +10075,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: s_mov_b64 s[44:45], exec
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: v_mov_b32_e32 v7, 0x20b0
+; GFX6-NEXT: v_mov_b32_e32 v7, 0x20a0
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, v7, s[40:43], 0 offen ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
@@ -10109,8 +10089,63 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; use s[8:15],s[16:23],s[24:31],s[0:7],s[36:39],vcc
; GFX6-NEXT: ;;#ASMEND
+; GFX6-NEXT: s_mov_b64 s[4:5], exec
+; GFX6-NEXT: s_mov_b64 exec, 15
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_mov_b32 s6, 0x82c00
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s6 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: v_readlane_b32 s0, v4, 0
+; GFX6-NEXT: v_readlane_b32 s1, v4, 1
+; GFX6-NEXT: v_readlane_b32 s2, v4, 2
+; GFX6-NEXT: v_readlane_b32 s3, v4, 3
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: s_mov_b64 exec, s[4:5]
+; GFX6-NEXT: s_mov_b32 s2, 0x86000
+; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s2, 0x86400
+; GFX6-NEXT: buffer_store_dword v12, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v13, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v14, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v15, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s2, 0x86800
+; GFX6-NEXT: buffer_store_dword v16, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v17, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v18, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v19, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: v_mov_b32_e32 v0, v20
+; GFX6-NEXT: v_mov_b32_e32 v1, v21
+; GFX6-NEXT: v_mov_b32_e32 v2, v22
+; GFX6-NEXT: v_mov_b32_e32 v3, v23
+; GFX6-NEXT: s_waitcnt expcnt(3)
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ;;#ASMEND
+; GFX6-NEXT: buffer_load_dword v16, off, s[40:43], s2 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt expcnt(2)
+; GFX6-NEXT: buffer_load_dword v17, off, s[40:43], s2 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt expcnt(1)
+; GFX6-NEXT: buffer_load_dword v18, off, s[40:43], s2 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v19, off, s[40:43], s2 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s2, 0x86400
+; GFX6-NEXT: buffer_load_dword v12, off, s[40:43], s2 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v13, off, s[40:43], s2 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v14, off, s[40:43], s2 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v15, off, s[40:43], s2 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s2, 0x86000
+; GFX6-NEXT: v_mov_b32_e32 v23, v3
+; GFX6-NEXT: v_mov_b32_e32 v22, v2
+; GFX6-NEXT: v_mov_b32_e32 v21, v1
+; GFX6-NEXT: v_mov_b32_e32 v20, v0
+; GFX6-NEXT: buffer_load_dword v0, off, s[40:43], s2 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Reload
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
@@ -10125,156 +10160,120 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: .LBB1_2: ; %ret
; GFX6-NEXT: s_or_b64 exec, exec, s[34:35]
-; GFX6-NEXT: s_mov_b64 s[4:5], exec
+; GFX6-NEXT: s_mov_b64 s[2:3], exec
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s6, 0x80400
+; GFX6-NEXT: s_mov_b32 s8, 0x80400
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s6 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s8 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: v_readlane_b32 s0, v4, 0
-; GFX6-NEXT: v_readlane_b32 s1, v4, 1
-; GFX6-NEXT: v_readlane_b32 s2, v4, 2
-; GFX6-NEXT: v_readlane_b32 s3, v4, 3
+; GFX6-NEXT: v_readlane_b32 s4, v4, 0
+; GFX6-NEXT: v_readlane_b32 s5, v4, 1
+; GFX6-NEXT: v_readlane_b32 s6, v4, 2
+; GFX6-NEXT: v_readlane_b32 s7, v4, 3
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[4:5]
-; GFX6-NEXT: s_mov_b64 s[36:37], s[0:1]
-; GFX6-NEXT: s_mov_b64 s[4:5], exec
-; GFX6-NEXT: s_mov_b64 exec, 15
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s6, 0x80800
+; GFX6-NEXT: s_mov_b64 exec, s[2:3]
+; GFX6-NEXT: s_mov_b32 s4, 0x85c00
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b64 s[2:3], s[6:7]
+; GFX6-NEXT: s_mov_b32 s4, 0x85800
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:240
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s6 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x85400
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: v_readlane_b32 s0, v4, 0
-; GFX6-NEXT: v_readlane_b32 s1, v4, 1
-; GFX6-NEXT: v_readlane_b32 s2, v4, 2
-; GFX6-NEXT: v_readlane_b32 s3, v4, 3
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:224
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x85000
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[4:5]
-; GFX6-NEXT: s_mov_b32 s0, 0x86800
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b64 s[38:39], s[2:3]
-; GFX6-NEXT: s_mov_b32 s0, 0x86400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:240
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x86000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:224
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85c00
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:208
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:192
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:176
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:160
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84c00
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:144
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:128
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:112
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:96
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:80
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83c00
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:64
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83400
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:208
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84c00
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:192
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84800
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:176
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84400
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:48
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:160
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83000
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84000
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:32
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:144
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83c00
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:16
-; GFX6-NEXT: buffer_store_dwordx4 v[0:3], v[5:6], s[36:39], 0 addr64
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:128
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83800
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:112
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83400
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:96
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83000
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:80
+; GFX6-NEXT: buffer_store_dwordx4 v[16:19], v[5:6], s[0:3], 0 addr64 offset:64
+; GFX6-NEXT: buffer_store_dwordx4 v[20:23], v[5:6], s[0:3], 0 addr64 offset:48
+; GFX6-NEXT: buffer_store_dwordx4 v[12:15], v[5:6], s[0:3], 0 addr64 offset:32
+; GFX6-NEXT: s_waitcnt expcnt(3)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:16
+; GFX6-NEXT: buffer_store_dwordx4 v[0:3], v[5:6], s[0:3], 0 addr64
; GFX6-NEXT: s_endpgm
;
; GFX9-FLATSCR-LABEL: test_limited_sgpr:
diff --git a/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir b/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir
index 3810fc46792fc6..2d21bd87757873 100644
--- a/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir
+++ b/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir
@@ -21,7 +21,7 @@ body: |
bb.2:
%4:vgpr_32 = PHI %5, %bb.3, %3, %bb.1
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
EXP_DONE 0, %4, %4, %4, %4, -1, 0, 15, implicit $exec
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir b/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir
index a3ebe94e046846..81d036b61aa8b5 100644
--- a/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir
+++ b/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir
@@ -57,7 +57,7 @@ body: |
S_BRANCH %bb.4
bb.3:
- SI_END_CF %8:sreg_64_xexec, implicit-def $exec, implicit-def $scc, implicit $exec
+ bb-prolog SI_END_CF %8:sreg_64_xexec, implicit-def $exec, implicit-def $scc, implicit $exec
%13:sreg_32 = S_MOV_B32 1
%15:vgpr_32 = COPY %13:sreg_32
%10:vgpr_32, dead %20:sreg_64_xexec = V_ADD_CO_U32_e64 %6:vgpr_32, %15:vgpr_32, 0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir b/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir
index 43b35d523e8c47..44db5cef5ddd3d 100644
--- a/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir
+++ b/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir
@@ -218,7 +218,7 @@ body: |
bb.18:
successors: %bb.20(0x40000000), %bb.19(0x40000000)
- $exec = S_OR_B64 $exec, %47, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %47, implicit-def $scc
%52:vgpr_32 = V_MAD_F32_e64 0, %3.sub1, 0, target-flags(amdgpu-gotprel32-lo) 0, 1, %3.sub0, 0, 0, implicit $mode, implicit $exec
%53:vgpr_32 = V_MUL_F32_e32 -2147483648, %3.sub1, implicit $mode, implicit $exec
%53:vgpr_32 = V_MAC_F32_e32 target-flags(amdgpu-gotprel32-hi) 1065353216, %3.sub2, %53, implicit $mode, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll b/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll
index 8390ef7782dd6c..87e7c186962f7b 100644
--- a/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll
+++ b/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll
@@ -121,7 +121,7 @@ define amdgpu_ps { <4 x float> } @test_return_to_epilog_with_optimized_kill(floa
; GCN-NEXT: successors: %bb.9(0x80000000)
; GCN-NEXT: liveins: $sgpr2_sgpr3
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; GCN-NEXT: S_BRANCH %bb.9
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.3.flow.preheader:
@@ -144,7 +144,7 @@ define amdgpu_ps { <4 x float> } @test_return_to_epilog_with_optimized_kill(floa
; GCN-NEXT: successors: %bb.6(0x40000000), %bb.2(0x40000000)
; GCN-NEXT: liveins: $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; GCN-NEXT: $sgpr2_sgpr3 = S_ANDN2_SAVEEXEC_B64 killed $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: S_CBRANCH_EXECZ %bb.2, implicit $exec
; GCN-NEXT: {{ $}}
@@ -160,7 +160,7 @@ define amdgpu_ps { <4 x float> } @test_return_to_epilog_with_optimized_kill(floa
; GCN-NEXT: liveins: $sgpr2_sgpr3, $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: $exec = S_MOV_B64 0
- ; GCN-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; GCN-NEXT: S_BRANCH %bb.9
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.8:
diff --git a/llvm/test/CodeGen/AMDGPU/trap-abis.ll b/llvm/test/CodeGen/AMDGPU/trap-abis.ll
index 5a359a5b2f3ccc..558f9c3e1a09c0 100644
--- a/llvm/test/CodeGen/AMDGPU/trap-abis.ll
+++ b/llvm/test/CodeGen/AMDGPU/trap-abis.ll
@@ -379,15 +379,14 @@ define amdgpu_kernel void @trap_with_use_after(ptr addrspace(1) %arg0, ptr addrs
; HSA-TRAP-GFX1100-O0-NEXT: scratch_store_b32 off, v0, off offset:4 ; 4-byte Folded Spill
; HSA-TRAP-GFX1100-O0-NEXT: s_cbranch_execnz .LBB2_2
; HSA-TRAP-GFX1100-O0-NEXT: ; %bb.1:
+; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v0, off, off offset:8 ; 4-byte Folded Reload
+; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v1, off, off offset:4 ; 4-byte Folded Reload
; HSA-TRAP-GFX1100-O0-NEXT: s_or_saveexec_b32 s6, -1
; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v2, off, off ; 4-byte Folded Reload
; HSA-TRAP-GFX1100-O0-NEXT: s_mov_b32 exec_lo, s6
; HSA-TRAP-GFX1100-O0-NEXT: s_waitcnt vmcnt(0)
; HSA-TRAP-GFX1100-O0-NEXT: v_readlane_b32 s0, v2, 0
; HSA-TRAP-GFX1100-O0-NEXT: v_readlane_b32 s1, v2, 1
-; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v0, off, off offset:8 ; 4-byte Folded Reload
-; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v1, off, off offset:4 ; 4-byte Folded Reload
-; HSA-TRAP-GFX1100-O0-NEXT: s_waitcnt vmcnt(0)
; HSA-TRAP-GFX1100-O0-NEXT: global_store_b32 v0, v1, s[0:1] dlc
; HSA-TRAP-GFX1100-O0-NEXT: s_waitcnt_vscnt null, 0x0
; HSA-TRAP-GFX1100-O0-NEXT: s_endpgm
diff --git a/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir b/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
index c70327aaea9938..ff77abc71301bf 100644
--- a/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
+++ b/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
@@ -68,8 +68,8 @@ body: |
successors:
liveins: $sgpr8_sgpr9_sgpr10_sgpr11, $sgpr13
- $sgpr0_sgpr1 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $sgpr0_sgpr1 = bb-prolog SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
bb.3:
liveins: $vgpr0, $vgpr1, $vgpr2, $sgpr2_sgpr3, $sgpr4_sgpr5
@@ -133,8 +133,8 @@ body: |
successors:
liveins: $sgpr8_sgpr9_sgpr10_sgpr11, $sgpr13
- $sgpr0_sgpr1 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $sgpr0_sgpr1 = bb-prolog SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
bb.3:
liveins: $vgpr0, $vgpr1, $vgpr2, $sgpr2_sgpr3, $sgpr4_sgpr5
diff --git a/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir b/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir
index 0192da03a3345d..5c54c9c805b171 100644
--- a/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir
@@ -57,7 +57,7 @@ body: |
bb.2:
successors: %bb.3(0x80000000)
- SI_END_CF %4:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%5:sreg_64 = V_CMP_GT_U32_e64 %0:vgpr_32, %1:sreg_32, implicit $exec
%6:sreg_64 = S_AND_B64 %5:sreg_64, $exec, implicit-def $scc
S_BRANCH %bb.3
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
index 323d898b5b2d9d..49da0b742534c1 100644
--- a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
@@ -155,7 +155,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, %mask, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask, implicit-def $scc
; CHECK-NEXT: S_BRANCH %bb.5
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
@@ -202,7 +202,7 @@ body: |
; CHECK-NEXT: bb.4:
; CHECK-NEXT: successors: %bb.2(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, %mask2, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask2, implicit-def $scc
; CHECK-NEXT: SI_SPILL_V32_SAVE [[V_MOV_B32_e32_4]], %stack.16, $sgpr32, 0, implicit $exec :: (store (s32) into %stack.16, addrspace 5)
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
@@ -270,7 +270,7 @@ body: |
bb.2: ; flow
successors: %bb.5(0x80000000)
- $exec_lo = S_OR_B32 $exec_lo, %mask, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask, implicit-def $scc
S_BRANCH %bb.5
bb.3: ; loop
@@ -298,7 +298,7 @@ body: |
bb.4: ; flow
successors: %bb.2(0x80000000)
- $exec_lo = S_OR_B32 $exec_lo, %mask2, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask2, implicit-def $scc
S_BRANCH %bb.2
bb.5: ; exit
diff --git a/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir b/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir
index 4ee427ad513905..f76ed27a6dbb82 100644
--- a/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir
+++ b/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir
@@ -150,7 +150,7 @@ body: |
S_CBRANCH_EXECZ %bb.4, implicit $exec
bb.1:
- V_CMPX_EQ_U32_nosdst_e32 0, killed $vgpr1, implicit-def $exec, implicit $exec
+ bb-prolog V_CMPX_EQ_U32_nosdst_e32 0, killed $vgpr1, implicit-def $exec, implicit $exec
S_CBRANCH_EXECZ %bb.3, implicit $exec
bb.2:
diff --git a/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll b/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll
index 51303f11a8f474..427577f21ca94f 100644
--- a/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll
+++ b/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll
@@ -58,20 +58,21 @@ define i32 @fix_sgpr_copies_indirect_call(ptr addrspace(5) %ptr) #0 {
; CHECK-NEXT: buffer_store_dword v41, off, s[0:3], s33 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
; CHECK-NEXT: .LBB0_2: ; =>This Inner Loop Header: Depth=1
+; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
+; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
; CHECK-NEXT: s_or_saveexec_b64 s[34:35], -1
; CHECK-NEXT: buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
-; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
-; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
-; CHECK-NEXT: s_waitcnt vmcnt(1)
+; CHECK-NEXT: s_waitcnt vmcnt(2)
; CHECK-NEXT: v_readfirstlane_b32 s6, v0
-; CHECK-NEXT: s_waitcnt vmcnt(0)
+; CHECK-NEXT: s_waitcnt vmcnt(1)
; CHECK-NEXT: v_readfirstlane_b32 s8, v1
; CHECK-NEXT: s_mov_b32 s4, s6
; CHECK-NEXT: s_mov_b32 s5, s8
; CHECK-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
; CHECK-NEXT: ; kill: def $sgpr6 killed $sgpr6 def $sgpr6_sgpr7
; CHECK-NEXT: s_mov_b32 s7, s8
+; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: v_writelane_b32 v41, s6, 14
; CHECK-NEXT: v_writelane_b32 v41, s7, 15
; CHECK-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -81,6 +82,7 @@ define i32 @fix_sgpr_copies_indirect_call(ptr addrspace(5) %ptr) #0 {
; CHECK-NEXT: buffer_store_dword v41, off, s[0:3], s33 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
; CHECK-NEXT: ; %bb.3: ; in Loop: Header=BB0_2 Depth=1
+; CHECK-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
; CHECK-NEXT: s_or_saveexec_b64 s[34:35], -1
; CHECK-NEXT: buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
@@ -99,7 +101,6 @@ define i32 @fix_sgpr_copies_indirect_call(ptr addrspace(5) %ptr) #0 {
; CHECK-NEXT: v_readlane_b32 s7, v41, 9
; CHECK-NEXT: v_readlane_b32 s4, v41, 10
; CHECK-NEXT: v_readlane_b32 s5, v41, 11
-; CHECK-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b64 s[22:23], s[2:3]
; CHECK-NEXT: s_mov_b64 s[20:21], s[0:1]
; CHECK-NEXT: s_mov_b64 s[0:1], s[20:21]
diff --git a/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir b/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir
index 5f3b543acf6a65..2cdebeb9075c1e 100644
--- a/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir
+++ b/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir
@@ -479,7 +479,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: liveins: $vcc_lo, $sgpr1, $vgpr0, $vgpr1
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_CNDMASK_B32_e64 0, $vgpr1, 0, $vgpr0, $vcc_lo, implicit $exec
; CHECK-NEXT: $exec_lo = S_XOR_B32 $vcc_lo, -1, implicit-def $scc
; CHECK-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit $vgpr0(tied-def 0) :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
@@ -490,7 +490,7 @@ body: |
successors: %bb.1, %bb.2
liveins: $vgpr0, $vgpr1
- renamable $vcc_lo = SI_WHOLE_WAVE_FUNC_SETUP implicit-def dead $exec, implicit $exec
+ renamable $vcc_lo = bb-prolog SI_WHOLE_WAVE_FUNC_SETUP implicit-def dead $exec, implicit $exec
$sgpr1 = S_MOV_B32 $exec_lo
V_CMPX_EQ_U32_nosdst_e64 $vgpr0, $vgpr1, implicit-def $exec, implicit $exec
S_CBRANCH_EXECZ %bb.2, implicit $exec
@@ -504,7 +504,7 @@ body: |
bb.2:
liveins: $vcc_lo, $sgpr1, $vgpr0, $vgpr1
- $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
renamable $vgpr0 = V_CNDMASK_B32_e64 0, $vgpr1, 0, $vgpr0, $vcc_lo, implicit $exec
SI_WHOLE_WAVE_FUNC_RETURN killed renamable $vcc_lo, implicit $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir
index 73e910e75fc193..1b80c82ae67139 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir
@@ -45,8 +45,8 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $vgpr0, $vgpr1, $m0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $exec_lo
- ; CHECK-NEXT: $exec_lo = S_WQM_B32 $exec_lo, implicit-def $scc
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = bb-prolog COPY $exec_lo
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_WQM_B32 $exec_lo, implicit-def $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr1
; CHECK-NEXT: [[DEF:%[0-9]+]]:sgpr_256 = IMPLICIT_DEF
@@ -58,7 +58,7 @@ body: |
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $m0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WQM:%[0-9]+]]:sreg_32 = ENTER_STRICT_WQM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WQM:%[0-9]+]]:sreg_32 = bb-prolog ENTER_STRICT_WQM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[DS_PARAM_LOAD:%[0-9]+]]:vgpr_32 = DS_PARAM_LOAD 0, 0, 0, 1, implicit $m0, implicit $exec
; CHECK-NEXT: $exec_lo = EXIT_STRICT_WQM [[ENTER_STRICT_WQM]]
; CHECK-NEXT: DBG_VALUE $noreg, $noreg
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir
index 604ab24bf02bcb..f66400b2704184 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir
@@ -67,7 +67,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: $exec = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir b/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir
index d77704857bba53..9b512c6220c361 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir
@@ -24,8 +24,8 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.8(0x40000000)
; CHECK-NEXT: liveins: $vgpr0, $vgpr1, $vgpr2
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $exec_lo
- ; CHECK-NEXT: $exec_lo = S_WQM_B32 $exec_lo, implicit-def $scc
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = bb-prolog COPY $exec_lo
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_WQM_B32 $exec_lo, implicit-def $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr1
; CHECK-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr0
@@ -79,7 +79,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.7(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[S_AND_B32_2:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[S_OR_SAVEEXEC_B32_]], implicit-def $scc
; CHECK-NEXT: $exec_lo = S_XOR_B32_term $exec_lo, [[S_AND_B32_2]], implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.7, implicit $exec
@@ -110,7 +110,7 @@ body: |
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.2(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_AND_SAVEEXEC_B32_1:%[0-9]+]]:sreg_32 = S_AND_SAVEEXEC_B32 [[COPY]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_AND_SAVEEXEC_B32_1:%[0-9]+]]:sreg_32 = bb-prolog S_AND_SAVEEXEC_B32 [[COPY]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[V_OR_B16_t16_e64_:%[0-9]+]].sub1:vreg_128 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: [[V_OR_B16_t16_e64_:%[0-9]+]].sub2:vreg_128 = COPY [[V_OR_B16_t16_e64_]].sub1
; CHECK-NEXT: [[V_OR_B16_t16_e64_:%[0-9]+]].sub3:vreg_128 = COPY [[V_OR_B16_t16_e64_]].sub1
@@ -123,16 +123,16 @@ body: |
; CHECK-NEXT: bb.6:
; CHECK-NEXT: successors: %bb.7(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY7]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY7]], implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.7:
; CHECK-NEXT: successors: %bb.8(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_AND_B32_2]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_AND_B32_2]], implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.8:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY4]], implicit-def $scc
- ; CHECK-NEXT: $exec_lo = S_AND_B32 $exec_lo, [[COPY]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY4]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_AND_B32 $exec_lo, [[COPY]], implicit-def $scc
; CHECK-NEXT: $vgpr0 = COPY [[V_MOV_B32_e32_]]
; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0
bb.0:
@@ -190,7 +190,7 @@ body: |
bb.2:
successors: %bb.3, %bb.7
- %24:sreg_32 = S_OR_SAVEEXEC_B32 %23, implicit-def $exec, implicit-def $scc, implicit $exec
+ %24:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 %23, implicit-def $exec, implicit-def $scc, implicit $exec
%25:sreg_32 = S_AND_B32 $exec_lo, %24, implicit-def $scc
$exec_lo = S_XOR_B32_term $exec_lo, %25, implicit-def $scc
S_CBRANCH_EXECZ %bb.7, implicit $exec
@@ -226,13 +226,13 @@ body: |
S_BRANCH %bb.2
bb.6:
- $exec_lo = S_OR_B32 $exec_lo, %29, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %29, implicit-def $scc
bb.7:
- $exec_lo = S_OR_B32 $exec_lo, %25, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %25, implicit-def $scc
bb.8:
- $exec_lo = S_OR_B32 $exec_lo, %6, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %6, implicit-def $scc
$vgpr0 = COPY %5
SI_RETURN_TO_EPILOG $vgpr0
...
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir b/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir
index c8b0c4ef68623c..b15842e59f31a9 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir
@@ -28,7 +28,7 @@ body: |
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $sgpr0, $vgpr0_vgpr1
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $exec_lo
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = bb-prolog COPY $exec_lo
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr0
; CHECK-NEXT: $exec_lo = S_WQM_B32 $exec_lo, implicit-def $scc
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vreg_64 = COPY $vgpr0_vgpr1
diff --git a/llvm/test/CodeGen/AMDGPU/wqm.mir b/llvm/test/CodeGen/AMDGPU/wqm.mir
index 4e5b5a6b43e7ef..de4b5017136888 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm.mir
@@ -6,6 +6,9 @@
define amdgpu_ps void @test_strict_wwm_scc() {
ret void
}
+ define amdgpu_ps void @test_strict_wwm_pseudo_probe() {
+ ret void
+ }
define amdgpu_ps void @test_strict_wwm_scc2() {
ret void
}
@@ -80,7 +83,7 @@ body: |
; CHECK-LABEL: name: test_strict_wwm_scc
; CHECK: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
; CHECK-NEXT: [[COPY2:%[0-9]+]]:sgpr_32 = COPY $sgpr1
@@ -105,6 +108,68 @@ body: |
...
+---
+# A pseudo probe at the block entry does not separate the exec mode switch from
+# the block start: it is still a block prolog instruction.
+name: test_strict_wwm_pseudo_probe
+alignment: 1
+exposesReturnsTwice: false
+legalized: false
+regBankSelected: false
+selected: false
+tracksRegLiveness: true
+registers:
+ - { id: 0, class: sgpr_32, preferred-register: '' }
+ - { id: 1, class: sgpr_32, preferred-register: '' }
+ - { id: 2, class: sgpr_32, preferred-register: '' }
+ - { id: 3, class: vgpr_32, preferred-register: '' }
+ - { id: 4, class: vgpr_32, preferred-register: '' }
+ - { id: 5, class: sgpr_32, preferred-register: '' }
+ - { id: 6, class: vgpr_32, preferred-register: '' }
+ - { id: 7, class: vgpr_32, preferred-register: '' }
+ - { id: 8, class: sreg_32_xm0, preferred-register: '' }
+ - { id: 9, class: sreg_32, preferred-register: '' }
+ - { id: 10, class: sreg_32, preferred-register: '' }
+ - { id: 11, class: vgpr_32, preferred-register: '' }
+ - { id: 12, class: vgpr_32, preferred-register: '' }
+liveins:
+ - { reg: '$sgpr0', virtual-reg: '%0' }
+ - { reg: '$sgpr1', virtual-reg: '%1' }
+ - { reg: '$sgpr2', virtual-reg: '%2' }
+ - { reg: '$vgpr0', virtual-reg: '%3' }
+body: |
+ bb.0:
+ liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
+
+ ; CHECK-LABEL: name: test_strict_wwm_pseudo_probe
+ ; CHECK: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: PSEUDO_PROBE 1, 1, 0, 0
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+ ; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
+ ; CHECK-NEXT: [[COPY2:%[0-9]+]]:sgpr_32 = COPY $sgpr1
+ ; CHECK-NEXT: [[COPY3:%[0-9]+]]:sgpr_32 = COPY $sgpr0
+ ; CHECK-NEXT: S_CMP_LT_I32 0, [[COPY3]], implicit-def $scc
+ ; CHECK-NEXT: [[V_ADD_CO_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_CO_U32_e32 [[COPY]], [[COPY]], implicit-def $vcc, implicit $exec
+ ; CHECK-NEXT: [[S_CSELECT_B32_:%[0-9]+]]:sgpr_32 = S_CSELECT_B32 [[COPY1]], [[COPY2]], implicit $scc
+ ; CHECK-NEXT: [[V_ADD_CO_U32_e32_1:%[0-9]+]]:vgpr_32 = V_ADD_CO_U32_e32 [[S_CSELECT_B32_]], [[V_ADD_CO_U32_e32_]], implicit-def $vcc, implicit $exec
+ ; CHECK-NEXT: $exec = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
+ ; CHECK-NEXT: early-clobber $vgpr0 = V_MOV_B32_e32 [[V_ADD_CO_U32_e32_1]], implicit $exec
+ ; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0
+ PSEUDO_PROBE 1, 1, 0, 0
+ %3 = COPY $vgpr0
+ %2 = COPY $sgpr2
+ %1 = COPY $sgpr1
+ %0 = COPY $sgpr0
+ S_CMP_LT_I32 0, %0, implicit-def $scc
+ %12 = V_ADD_CO_U32_e32 %3, %3, implicit-def $vcc, implicit $exec
+ %5 = S_CSELECT_B32 %2, %1, implicit $scc
+ %11 = V_ADD_CO_U32_e32 %5, %12, implicit-def $vcc, implicit $exec
+ $vgpr0 = STRICT_WWM %11, implicit $exec
+ SI_RETURN_TO_EPILOG $vgpr0
+
+...
---
# Second test for awareness that s_or_saveexec_b64 clobbers SCC
# Because entry block is treated differently.
@@ -116,7 +181,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: $exec = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
@@ -298,7 +363,7 @@ body: |
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $sgpr1, $sgpr2, $vgpr1, $vgpr2
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = COPY $exec
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = bb-prolog COPY $exec
; CHECK-NEXT: $m0 = COPY $sgpr1
; CHECK-NEXT: $exec = S_WQM_B64 $exec, implicit-def $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr1
@@ -417,8 +482,8 @@ body: |
; CHECK: bb.0:
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = COPY $exec
- ; CHECK-NEXT: $exec = S_WQM_B64 $exec, implicit-def $scc
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = bb-prolog COPY $exec
+ ; CHECK-NEXT: $exec = bb-prolog S_WQM_B64 $exec, implicit-def $scc
; CHECK-NEXT: undef [[V_MOV_B32_e32_:%[0-9]+]].sub0:vreg_64 = V_MOV_B32_e32 -10, implicit $exec
; CHECK-NEXT: [[V_MOV_B32_e32_:%[0-9]+]].sub1:vreg_64 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: [[S_GETPC_B64_:%[0-9]+]]:sreg_64 = S_GETPC_B64
@@ -444,7 +509,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.4:
- ; CHECK-NEXT: $exec = S_AND_B64 $exec, [[COPY]], implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_AND_B64 $exec, [[COPY]], implicit-def $scc
; CHECK-NEXT: [[DEF:%[0-9]+]]:sgpr_128 = IMPLICIT_DEF
; CHECK-NEXT: [[IMAGE_SAMPLE_V4_V2_:%[0-9]+]]:vreg_128 = IMAGE_SAMPLE_V4_V2 [[V_MOV_B32_e32_]], [[S_LOAD_DWORDX8_IMM]], [[DEF]], 15, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7)
; CHECK-NEXT: $vgpr0 = COPY [[IMAGE_SAMPLE_V4_V2_]].sub0
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir b/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir
index 8647c6314c88eb..87874efd16a681 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir
+++ b/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir
@@ -30,8 +30,8 @@ machineFunctionInfo:
body: |
bb.0:
liveins: $sgpr34, $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, $sgpr68_sgpr69_sgpr70_sgpr71_sgpr72_sgpr73_sgpr74_sgpr75_sgpr76_sgpr77_sgpr78_sgpr79_sgpr80_sgpr81_sgpr82_sgpr83_sgpr84_sgpr85_sgpr86_sgpr87_sgpr88_sgpr89_sgpr90_sgpr91_sgpr92_sgpr93_sgpr94_sgpr95_sgpr96_sgpr97_sgpr98_sgpr99
- %0:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = EXIT_STRICT_WWM killed %0
+ %0:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog EXIT_STRICT_WWM killed %0
SI_SPILL_S1024_SAVE killed $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, %stack.0, implicit $exec, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $sgpr32
SI_SPILL_S1024_SAVE killed $sgpr68_sgpr69_sgpr70_sgpr71_sgpr72_sgpr73_sgpr74_sgpr75_sgpr76_sgpr77_sgpr78_sgpr79_sgpr80_sgpr81_sgpr82_sgpr83_sgpr84_sgpr85_sgpr86_sgpr87_sgpr88_sgpr89_sgpr90_sgpr91_sgpr92_sgpr93_sgpr94_sgpr95_sgpr96_sgpr97_sgpr98_sgpr99, %stack.1, implicit $exec, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $sgpr32
SI_SPILL_S32_SAVE killed $sgpr34, %stack.2, implicit $exec, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $sgpr32
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll b/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
index 9989907f4b9b98..dbefc926c15108 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
+++ b/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
@@ -212,12 +212,12 @@ define amdgpu_gfx void @strict_wwm_cfg(ptr addrspace(8) inreg %tmp14, i32 %arg)
; GFX9-O0-NEXT: v_readlane_b32 s36, v5, 4
; GFX9-O0-NEXT: v_readlane_b32 s37, v5, 5
; GFX9-O0-NEXT: s_or_b64 exec, exec, s[36:37]
+; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: v_readlane_b32 s38, v5, 0
; GFX9-O0-NEXT: v_readlane_b32 s39, v5, 1
; GFX9-O0-NEXT: v_readlane_b32 s34, v5, 2
; GFX9-O0-NEXT: v_readlane_b32 s35, v5, 3
-; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_cmp_eq_u32_e64 s[36:37], v0, v3
; GFX9-O0-NEXT: v_cndmask_b32_e64 v0, 0, -1, s[36:37]
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll b/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll
index 0ef5725607920a..c14cd487c7d56a 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll
+++ b/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll
@@ -189,12 +189,12 @@ define amdgpu_cs void @cfg(ptr addrspace(8) inreg %tmp14, i32 %arg) {
; GFX9-O0-NEXT: v_readlane_b32 s4, v5, 5
; GFX9-O0-NEXT: v_readlane_b32 s5, v5, 6
; GFX9-O0-NEXT: s_or_b64 exec, exec, s[4:5]
+; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: v_readlane_b32 s2, v5, 1
; GFX9-O0-NEXT: v_readlane_b32 s3, v5, 2
; GFX9-O0-NEXT: v_readlane_b32 s0, v5, 3
; GFX9-O0-NEXT: v_readlane_b32 s1, v5, 4
-; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], v0, v3
; GFX9-O0-NEXT: v_cndmask_b32_e64 v0, 0, -1, s[4:5]
@@ -1052,12 +1052,12 @@ define amdgpu_cs void @strict_wwm_cfg(ptr addrspace(8) inreg %tmp14, i32 %arg) {
; GFX9-O0-NEXT: v_readlane_b32 s4, v5, 5
; GFX9-O0-NEXT: v_readlane_b32 s5, v5, 6
; GFX9-O0-NEXT: s_or_b64 exec, exec, s[4:5]
+; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: v_readlane_b32 s2, v5, 1
; GFX9-O0-NEXT: v_readlane_b32 s3, v5, 2
; GFX9-O0-NEXT: v_readlane_b32 s0, v5, 3
; GFX9-O0-NEXT: v_readlane_b32 s1, v5, 4
-; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], v0, v3
; GFX9-O0-NEXT: v_cndmask_b32_e64 v0, 0, -1, s[4:5]
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir b/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir
index ce8aebe7458ed8..3d14dcf053af7a 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir
+++ b/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir
@@ -73,7 +73,7 @@ body: |
bb.7:
liveins: $sgpr60_sgpr61
- $exec = S_OR_B64 $exec, killed $sgpr60_sgpr61, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr60_sgpr61, implicit-def $scc
$sgpr2_sgpr3 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr32
S_BRANCH %bb.5
>From 7eec753505b838767165f56789612e0ab2fa73e1 Mon Sep 17 00:00:00 2001
From: Gabriel Baraldi <baraldigabriel at gmail.com>
Date: Tue, 22 Sep 2026 15:24:18 +0000
Subject: [PATCH 3/3] [AMDGPU] Regenerate amdgcn.bitcast tests for the BBProlog
flag
Register assignment shifts because sgpr-regalloc split copies now follow
the exec restore in Flow blocks. Pure regeneration with
update_llc_test_checks.py; s_or_saveexec/s_andn2_saveexec counts are
unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply at anthropic.com>
---
.../CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll | 13540 ++++++++--------
.../CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll | 2023 +--
.../CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll | 684 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll | 1892 ++-
.../CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll | 2811 ++--
.../CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll | 4860 +++---
.../CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll | 6734 ++++----
.../CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll | 7506 +++++----
8 files changed, 20982 insertions(+), 19068 deletions(-)
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
index 47468f3c7c4cf4..73e8e641da0875 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
@@ -22760,139 +22760,139 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: v_writelane_b32 v33, s55, 13
; SI-NEXT: v_writelane_b32 v33, s64, 14
; SI-NEXT: v_writelane_b32 v33, s65, 15
-; SI-NEXT: v_writelane_b32 v33, s66, 16
-; SI-NEXT: v_writelane_b32 v33, s67, 17
-; SI-NEXT: v_writelane_b32 v33, s68, 18
-; SI-NEXT: v_writelane_b32 v33, s69, 19
-; SI-NEXT: v_writelane_b32 v33, s70, 20
-; SI-NEXT: v_writelane_b32 v33, s71, 21
-; SI-NEXT: v_writelane_b32 v33, s80, 22
-; SI-NEXT: v_writelane_b32 v33, s81, 23
-; SI-NEXT: v_writelane_b32 v33, s82, 24
-; SI-NEXT: v_writelane_b32 v33, s83, 25
-; SI-NEXT: v_writelane_b32 v33, s84, 26
-; SI-NEXT: v_writelane_b32 v33, s85, 27
-; SI-NEXT: v_writelane_b32 v33, s86, 28
-; SI-NEXT: v_writelane_b32 v33, s87, 29
-; SI-NEXT: v_writelane_b32 v33, s96, 30
-; SI-NEXT: v_writelane_b32 v33, s97, 31
-; SI-NEXT: v_writelane_b32 v33, s98, 32
-; SI-NEXT: v_writelane_b32 v33, s30, 33
-; SI-NEXT: v_writelane_b32 v33, s31, 34
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_readfirstlane_b32 s4, v17
-; SI-NEXT: v_readfirstlane_b32 s30, v16
-; SI-NEXT: v_readfirstlane_b32 s31, v15
-; SI-NEXT: v_readfirstlane_b32 s34, v14
-; SI-NEXT: v_readfirstlane_b32 s35, v13
-; SI-NEXT: v_readfirstlane_b32 s36, v12
-; SI-NEXT: v_readfirstlane_b32 s37, v11
-; SI-NEXT: v_readfirstlane_b32 s38, v10
-; SI-NEXT: v_readfirstlane_b32 s39, v9
-; SI-NEXT: v_readfirstlane_b32 s48, v8
-; SI-NEXT: v_readfirstlane_b32 s49, v7
-; SI-NEXT: v_readfirstlane_b32 s50, v6
-; SI-NEXT: v_readfirstlane_b32 s51, v5
-; SI-NEXT: v_readfirstlane_b32 s52, v4
-; SI-NEXT: v_readfirstlane_b32 s53, v3
-; SI-NEXT: v_readfirstlane_b32 s54, v2
-; SI-NEXT: v_readfirstlane_b32 s55, v1
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_readfirstlane_b32 s64, v0
+; SI-NEXT: v_writelane_b32 v33, s30, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 17
+; SI-NEXT: v_readfirstlane_b32 s7, v17
; SI-NEXT: ; implicit-def: $vgpr34 : SGPR spill to VGPR lane
+; SI-NEXT: v_readfirstlane_b32 s9, v16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v34, s4, 0
+; SI-NEXT: v_writelane_b32 v34, s7, 0
+; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: v_writelane_b32 v34, s9, 1
+; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v34, s11, 2
+; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v34, s13, 3
+; SI-NEXT: v_readfirstlane_b32 s41, v12
+; SI-NEXT: v_writelane_b32 v34, s15, 4
+; SI-NEXT: v_readfirstlane_b32 s43, v11
+; SI-NEXT: v_writelane_b32 v34, s41, 5
+; SI-NEXT: v_readfirstlane_b32 s45, v10
+; SI-NEXT: v_writelane_b32 v34, s43, 6
+; SI-NEXT: v_readfirstlane_b32 s47, v9
+; SI-NEXT: v_writelane_b32 v34, s45, 7
+; SI-NEXT: v_readfirstlane_b32 s57, v8
+; SI-NEXT: v_writelane_b32 v34, s47, 8
+; SI-NEXT: v_readfirstlane_b32 s59, v7
+; SI-NEXT: v_writelane_b32 v34, s57, 9
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_writelane_b32 v34, s59, 10
+; SI-NEXT: v_readfirstlane_b32 s63, v5
+; SI-NEXT: v_writelane_b32 v34, s61, 11
+; SI-NEXT: v_readfirstlane_b32 s73, v4
+; SI-NEXT: v_writelane_b32 v34, s63, 12
+; SI-NEXT: v_readfirstlane_b32 s6, v3
+; SI-NEXT: v_writelane_b32 v34, s73, 13
+; SI-NEXT: v_readfirstlane_b32 s5, v2
+; SI-NEXT: v_writelane_b32 v34, s6, 14
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: v_readfirstlane_b32 s8, v18
+; SI-NEXT: v_writelane_b32 v34, s5, 15
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_readfirstlane_b32 s8, v0
+; SI-NEXT: v_writelane_b32 v34, s4, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 17
; SI-NEXT: s_cbranch_scc0 .LBB17_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s8, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s4, 16
-; SI-NEXT: s_and_b32 s4, s50, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 2
-; SI-NEXT: s_lshl_b32 s4, s50, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 1
-; SI-NEXT: s_and_b32 s4, s51, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 4
-; SI-NEXT: s_lshl_b32 s4, s51, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 3
-; SI-NEXT: s_and_b32 s4, s52, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 6
-; SI-NEXT: s_lshl_b32 s4, s52, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 5
-; SI-NEXT: s_and_b32 s4, s53, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 8
-; SI-NEXT: s_lshl_b32 s4, s53, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 7
-; SI-NEXT: s_and_b32 s4, s55, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 10
-; SI-NEXT: s_lshl_b32 s4, s55, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 9
-; SI-NEXT: s_and_b32 s4, s64, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 12
-; SI-NEXT: s_lshl_b32 s4, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 11
-; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 14
-; SI-NEXT: s_lshl_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 13
-; SI-NEXT: s_and_b32 s4, s28, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 16
-; SI-NEXT: s_lshl_b32 s4, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 15
+; SI-NEXT: s_and_b32 s88, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s4, 16
; SI-NEXT: s_and_b32 s4, s27, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 18
+; SI-NEXT: v_writelane_b32 v34, s4, 19
; SI-NEXT: s_lshl_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 17
+; SI-NEXT: v_writelane_b32 v34, s4, 18
; SI-NEXT: s_and_b32 s4, s26, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: v_writelane_b32 v34, s4, 21
; SI-NEXT: s_lshl_b32 s4, s26, 16
-; SI-NEXT: s_and_b32 s10, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s30, 16
-; SI-NEXT: s_and_b32 s12, s31, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s31, 16
-; SI-NEXT: s_and_b32 s14, s34, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s34, 16
-; SI-NEXT: s_and_b32 s40, s35, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s35, 16
-; SI-NEXT: s_and_b32 s42, s36, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s36, 16
-; SI-NEXT: s_and_b32 s44, s37, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s37, 16
-; SI-NEXT: s_and_b32 s46, s38, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s38, 16
-; SI-NEXT: s_and_b32 s56, s39, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s39, 16
-; SI-NEXT: s_and_b32 s58, s48, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s48, 16
-; SI-NEXT: s_and_b32 s60, s49, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s49, 16
-; SI-NEXT: s_and_b32 s78, s54, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s54, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 19
-; SI-NEXT: s_and_b32 s62, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s25, 16
-; SI-NEXT: s_and_b32 s72, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s24, 16
-; SI-NEXT: s_and_b32 s74, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s23, 16
-; SI-NEXT: s_and_b32 s76, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s75, s22, 16
-; SI-NEXT: s_and_b32 s88, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s79, s21, 16
-; SI-NEXT: s_and_b32 s90, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s89, s20, 16
-; SI-NEXT: s_and_b32 s92, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s91, s19, 16
-; SI-NEXT: s_and_b32 s94, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s93, s18, 16
-; SI-NEXT: s_and_b32 vcc_lo, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s95, s17, 16
-; SI-NEXT: s_and_b32 s6, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_hi, s16, 16
+; SI-NEXT: s_and_b32 s55, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_and_b32 s10, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_and_b32 s12, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_and_b32 s14, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_and_b32 s40, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_and_b32 s42, s41, 0xffff0000
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_and_b32 s44, s43, 0xffff0000
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s46, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_and_b32 s56, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_and_b32 s58, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_and_b32 s60, s59, 0xffff0000
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_and_b32 s62, s61, 0xffff0000
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_and_b32 s72, s63, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_and_b32 s74, s73, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s73, 16
+; SI-NEXT: s_and_b32 s76, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s75, s6, 16
+; SI-NEXT: s_and_b32 s78, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s5, 16
+; SI-NEXT: s_and_b32 s90, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s8, 16
+; SI-NEXT: s_and_b32 s92, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s29, 16
+; SI-NEXT: s_and_b32 s94, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s28, 16
+; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s6, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s8, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s16, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB17_3
; SI-NEXT: .LBB17_2:
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr50
+; SI-NEXT: ; implicit-def: $sgpr49
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr95
@@ -22905,6 +22905,8 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr88
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr75
; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $sgpr73
@@ -22913,8 +22915,6 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr61
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr59
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $sgpr57
@@ -22936,39 +22936,7 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
; SI-NEXT: ; implicit-def: $sgpr4
@@ -22980,70 +22948,79 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s65, s63
-; SI-NEXT: s_mov_b32 s66, s72
-; SI-NEXT: s_mov_b32 s67, s73
-; SI-NEXT: s_mov_b32 s68, s74
-; SI-NEXT: s_mov_b32 s69, s75
-; SI-NEXT: s_mov_b32 s70, s76
-; SI-NEXT: s_mov_b32 s71, s79
-; SI-NEXT: s_mov_b32 s80, s88
-; SI-NEXT: s_mov_b32 s81, s89
-; SI-NEXT: s_mov_b32 s82, s90
-; SI-NEXT: s_mov_b32 s83, s91
-; SI-NEXT: s_mov_b32 s84, s92
-; SI-NEXT: s_mov_b32 s85, s93
-; SI-NEXT: s_mov_b32 s86, s94
-; SI-NEXT: s_mov_b32 s87, s95
-; SI-NEXT: s_mov_b32 s96, vcc_lo
-; SI-NEXT: s_mov_b32 vcc_lo, s61
-; SI-NEXT: s_mov_b32 s97, vcc_hi
-; SI-NEXT: s_mov_b32 vcc_hi, s62
-; SI-NEXT: s_mov_b32 s98, s6
+; SI-NEXT: s_mov_b32 s64, s30
+; SI-NEXT: s_mov_b32 s30, s6
+; SI-NEXT: s_mov_b32 s65, s8
; SI-NEXT: s_mov_b32 s4, s7
-; SI-NEXT: s_mov_b32 s5, s8
-; SI-NEXT: s_mov_b32 s7, s9
-; SI-NEXT: s_mov_b32 s8, s10
-; SI-NEXT: s_mov_b32 s9, s11
-; SI-NEXT: s_mov_b32 s10, s12
-; SI-NEXT: s_mov_b32 s11, s13
-; SI-NEXT: s_mov_b32 s12, s14
-; SI-NEXT: s_mov_b32 s13, s15
-; SI-NEXT: s_mov_b32 s14, s40
-; SI-NEXT: s_mov_b32 s15, s41
-; SI-NEXT: s_mov_b32 s40, s42
-; SI-NEXT: s_mov_b32 s41, s43
-; SI-NEXT: s_mov_b32 s42, s44
-; SI-NEXT: s_mov_b32 s43, s45
-; SI-NEXT: s_mov_b32 s44, s46
-; SI-NEXT: s_mov_b32 s45, s47
-; SI-NEXT: s_mov_b32 s46, s56
-; SI-NEXT: s_mov_b32 s47, s57
-; SI-NEXT: s_mov_b32 s56, s58
-; SI-NEXT: s_mov_b32 s57, s59
-; SI-NEXT: s_mov_b32 s58, s60
-; SI-NEXT: v_readlane_b32 s59, v34, 1
-; SI-NEXT: v_readlane_b32 s60, v34, 2
-; SI-NEXT: v_readlane_b32 s61, v34, 3
-; SI-NEXT: v_readlane_b32 s62, v34, 4
-; SI-NEXT: v_readlane_b32 s63, v34, 5
-; SI-NEXT: v_readlane_b32 s72, v34, 6
-; SI-NEXT: v_readlane_b32 s73, v34, 7
-; SI-NEXT: v_readlane_b32 s74, v34, 8
-; SI-NEXT: v_readlane_b32 s6, v34, 9
-; SI-NEXT: v_readlane_b32 s75, v34, 10
-; SI-NEXT: v_readlane_b32 s76, v34, 11
-; SI-NEXT: v_readlane_b32 s79, v34, 12
-; SI-NEXT: v_readlane_b32 s88, v34, 13
-; SI-NEXT: v_readlane_b32 s89, v34, 14
-; SI-NEXT: v_readlane_b32 s90, v34, 15
-; SI-NEXT: v_readlane_b32 s91, v34, 16
-; SI-NEXT: v_readlane_b32 s92, v34, 17
-; SI-NEXT: v_readlane_b32 s93, v34, 18
-; SI-NEXT: v_readlane_b32 s94, v34, 19
-; SI-NEXT: v_readlane_b32 s95, v34, 20
+; SI-NEXT: s_mov_b32 s5, s55
+; SI-NEXT: s_mov_b32 s55, s9
+; SI-NEXT: s_mov_b32 s9, s10
+; SI-NEXT: s_mov_b32 s10, s11
+; SI-NEXT: s_mov_b32 s11, s12
+; SI-NEXT: s_mov_b32 s12, s13
+; SI-NEXT: s_mov_b32 s13, s14
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s40, s41
+; SI-NEXT: s_mov_b32 s41, s42
+; SI-NEXT: s_mov_b32 s42, s43
+; SI-NEXT: s_mov_b32 s43, s44
+; SI-NEXT: s_mov_b32 s44, s45
+; SI-NEXT: s_mov_b32 s45, s46
+; SI-NEXT: s_mov_b32 s46, s47
+; SI-NEXT: s_mov_b32 s47, s56
+; SI-NEXT: s_mov_b32 s56, s57
+; SI-NEXT: s_mov_b32 s57, s58
+; SI-NEXT: s_mov_b32 s58, s59
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_mov_b32 s60, s61
+; SI-NEXT: s_mov_b32 s61, s62
+; SI-NEXT: s_mov_b32 s62, s63
+; SI-NEXT: s_mov_b32 s63, s72
+; SI-NEXT: s_mov_b32 s72, s73
+; SI-NEXT: s_mov_b32 s73, s74
+; SI-NEXT: s_mov_b32 s74, s75
+; SI-NEXT: s_mov_b32 s75, s76
+; SI-NEXT: v_readlane_b32 s6, v34, 18
+; SI-NEXT: v_readlane_b32 s7, v34, 19
+; SI-NEXT: v_readlane_b32 s8, v34, 20
+; SI-NEXT: v_readlane_b32 s76, v34, 21
; SI-NEXT: s_cbranch_scc1 .LBB17_5
; SI-NEXT: ; %bb.4: ; %cmp.true
+; SI-NEXT: v_readlane_b32 s4, v34, 17
+; SI-NEXT: s_add_i32 s6, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 16
+; SI-NEXT: s_add_i32 s7, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 15
+; SI-NEXT: s_add_i32 s8, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 14
+; SI-NEXT: s_add_i32 s74, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 13
+; SI-NEXT: s_add_i32 s72, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 12
+; SI-NEXT: s_add_i32 s62, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 11
+; SI-NEXT: s_add_i32 s60, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 10
+; SI-NEXT: s_add_i32 s58, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 9
+; SI-NEXT: s_add_i32 s56, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 8
+; SI-NEXT: s_add_i32 s46, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 7
+; SI-NEXT: s_add_i32 s44, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 6
+; SI-NEXT: s_add_i32 s42, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 5
+; SI-NEXT: s_add_i32 s40, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 4
+; SI-NEXT: s_add_i32 s14, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 3
+; SI-NEXT: s_add_i32 s12, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 2
+; SI-NEXT: s_add_i32 s10, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 1
+; SI-NEXT: s_add_i32 s11, s4, 3
; SI-NEXT: v_readlane_b32 s4, v34, 0
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_add_i32 s17, s17, 3
@@ -23059,236 +23036,202 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_add_i32 s64, s64, 3
-; SI-NEXT: s_add_i32 s55, s55, 3
-; SI-NEXT: s_add_i32 s54, s54, 3
-; SI-NEXT: s_add_i32 s53, s53, 3
-; SI-NEXT: s_add_i32 s52, s52, 3
-; SI-NEXT: s_add_i32 s51, s51, 3
-; SI-NEXT: s_add_i32 s50, s50, 3
-; SI-NEXT: s_add_i32 s49, s49, 3
-; SI-NEXT: s_add_i32 s48, s48, 3
-; SI-NEXT: s_add_i32 s39, s39, 3
-; SI-NEXT: s_add_i32 s38, s38, 3
-; SI-NEXT: s_add_i32 s37, s37, 3
-; SI-NEXT: s_add_i32 s36, s36, 3
-; SI-NEXT: s_add_i32 s35, s35, 3
-; SI-NEXT: s_add_i32 s34, s34, 3
-; SI-NEXT: s_add_i32 s31, s31, 3
-; SI-NEXT: s_add_i32 s30, s30, 3
; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_and_b32 s8, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_and_b32 s10, s31, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s31, 16
-; SI-NEXT: s_and_b32 s12, s34, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s34, 16
-; SI-NEXT: s_and_b32 s14, s35, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s35, 16
-; SI-NEXT: s_and_b32 s40, s36, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s36, 16
-; SI-NEXT: s_and_b32 s42, s37, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s37, 16
-; SI-NEXT: s_and_b32 s44, s38, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s38, 16
-; SI-NEXT: s_and_b32 s46, s39, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s39, 16
-; SI-NEXT: s_and_b32 s56, s48, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s48, 16
-; SI-NEXT: s_and_b32 s58, s49, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s49, 16
-; SI-NEXT: s_and_b32 s60, s50, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s50, 16
-; SI-NEXT: s_and_b32 s62, s51, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s51, 16
-; SI-NEXT: s_and_b32 s72, s52, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s52, 16
-; SI-NEXT: s_and_b32 s74, s53, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s53, 16
-; SI-NEXT: s_and_b32 s78, s54, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s54, 16
-; SI-NEXT: s_and_b32 s75, s55, 0xffff0000
-; SI-NEXT: s_lshl_b32 s6, s55, 16
-; SI-NEXT: s_and_b32 s79, s64, 0xffff0000
-; SI-NEXT: s_lshl_b32 s76, s64, 16
-; SI-NEXT: s_and_b32 s89, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s88, s29, 16
-; SI-NEXT: s_and_b32 s91, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s28, 16
-; SI-NEXT: s_and_b32 s93, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s92, s27, 16
-; SI-NEXT: s_and_b32 s95, s26, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s26, 16
-; SI-NEXT: s_and_b32 vcc_hi, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s25, 16
-; SI-NEXT: s_and_b32 s66, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s65, s24, 16
-; SI-NEXT: s_and_b32 s68, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s23, 16
-; SI-NEXT: s_and_b32 s70, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s22, 16
-; SI-NEXT: s_and_b32 s80, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s21, 16
-; SI-NEXT: s_and_b32 s82, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s20, 16
-; SI-NEXT: s_and_b32 s84, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s83, s19, 16
-; SI-NEXT: s_and_b32 s86, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s85, s18, 16
-; SI-NEXT: s_and_b32 s96, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s87, s17, 16
-; SI-NEXT: s_and_b32 s98, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s97, s16, 16
+; SI-NEXT: s_and_b32 s9, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s55, s11, 16
+; SI-NEXT: s_and_b32 s11, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s13, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_and_b32 s15, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_and_b32 s41, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s43, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s45, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_and_b32 s47, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_and_b32 s57, s56, 0xffff0000
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_and_b32 s59, s58, 0xffff0000
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_and_b32 s61, s60, 0xffff0000
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_and_b32 s63, s62, 0xffff0000
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_and_b32 s73, s72, 0xffff0000
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_and_b32 s75, s74, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s74, 16
+; SI-NEXT: s_and_b32 s78, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s8, 16
+; SI-NEXT: s_and_b32 s88, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s7, 16
+; SI-NEXT: s_and_b32 s90, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s6, 16
+; SI-NEXT: s_and_b32 s92, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s29, 16
+; SI-NEXT: s_and_b32 s94, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s28, 16
+; SI-NEXT: s_and_b32 s7, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s6, s27, 16
+; SI-NEXT: s_and_b32 s76, s26, 0xffff0000
+; SI-NEXT: s_lshl_b32 s8, s26, 16
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s30, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s65, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s16, 16
; SI-NEXT: .LBB17_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s98
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s97
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s64
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s96
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s31
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s34
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s36
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s38
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s48
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s49
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s50
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s52
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s54
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s30
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, vcc_hi
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_hi
-; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_lo
+; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s95
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s76
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s8
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s7
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s6
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s93
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s91
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s90
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s89
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s79
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s77
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s75
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s73
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s63
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s61
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s59
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s57
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s47
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s45
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s15
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s13
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s55
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
-; SI-NEXT: v_readlane_b32 s30, v33, 33
+; SI-NEXT: v_readlane_b32 s30, v33, 16
; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
-; SI-NEXT: v_readlane_b32 s31, v33, 34
-; SI-NEXT: v_readlane_b32 s98, v33, 32
-; SI-NEXT: v_readlane_b32 s97, v33, 31
-; SI-NEXT: v_readlane_b32 s96, v33, 30
-; SI-NEXT: v_readlane_b32 s87, v33, 29
-; SI-NEXT: v_readlane_b32 s86, v33, 28
-; SI-NEXT: v_readlane_b32 s85, v33, 27
-; SI-NEXT: v_readlane_b32 s84, v33, 26
-; SI-NEXT: v_readlane_b32 s83, v33, 25
-; SI-NEXT: v_readlane_b32 s82, v33, 24
-; SI-NEXT: v_readlane_b32 s81, v33, 23
-; SI-NEXT: v_readlane_b32 s80, v33, 22
-; SI-NEXT: v_readlane_b32 s71, v33, 21
-; SI-NEXT: v_readlane_b32 s70, v33, 20
-; SI-NEXT: v_readlane_b32 s69, v33, 19
-; SI-NEXT: v_readlane_b32 s68, v33, 18
-; SI-NEXT: v_readlane_b32 s67, v33, 17
-; SI-NEXT: v_readlane_b32 s66, v33, 16
+; SI-NEXT: v_readlane_b32 s31, v33, 17
; SI-NEXT: v_readlane_b32 s65, v33, 15
; SI-NEXT: v_readlane_b32 s64, v33, 14
; SI-NEXT: v_readlane_b32 s55, v33, 13
@@ -32495,183 +32438,213 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB23_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB23_3
; SI-NEXT: .LBB23_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB23_3: ; %Flow
@@ -32680,9 +32653,9 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB23_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -32694,7 +32667,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -32702,11 +32675,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -32717,7 +32690,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -32726,11 +32699,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -32741,7 +32714,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -32750,11 +32723,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -32765,7 +32738,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -32774,11 +32747,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -32786,47 +32759,56 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -32834,10 +32816,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -32845,12 +32828,14 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -32858,10 +32843,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -32869,12 +32855,14 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -32882,10 +32870,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -32893,12 +32882,14 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -32906,10 +32897,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -32925,7 +32917,8 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -32973,24 +32966,8 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB23_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -35539,195 +35516,215 @@ define inreg <32 x i32> @bitcast_v64i16_to_v32i32_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB27_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB27_3
; SI-NEXT: .LBB27_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB27_3: ; %Flow
@@ -35738,168 +35735,188 @@ define inreg <32 x i32> @bitcast_v64i16_to_v32i32_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB27_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -35932,23 +35949,7 @@ define inreg <32 x i32> @bitcast_v64i16_to_v32i32_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -43435,224 +43436,283 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_writelane_b32 v63, s30, 34
; SI-NEXT: v_writelane_b32 v63, s31, 35
; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: v_readfirstlane_b32 s9, v18
-; SI-NEXT: v_readfirstlane_b32 s8, v17
-; SI-NEXT: v_readfirstlane_b32 s31, v16
-; SI-NEXT: v_readfirstlane_b32 s30, v15
-; SI-NEXT: v_readfirstlane_b32 s35, v14
-; SI-NEXT: v_readfirstlane_b32 s34, v13
-; SI-NEXT: v_readfirstlane_b32 s37, v12
-; SI-NEXT: v_readfirstlane_b32 s36, v11
-; SI-NEXT: v_readfirstlane_b32 s39, v10
-; SI-NEXT: v_readfirstlane_b32 s38, v9
-; SI-NEXT: v_readfirstlane_b32 s49, v8
-; SI-NEXT: v_readfirstlane_b32 s48, v7
-; SI-NEXT: v_readfirstlane_b32 s51, v6
-; SI-NEXT: v_readfirstlane_b32 s50, v5
-; SI-NEXT: v_readfirstlane_b32 s53, v4
-; SI-NEXT: v_readfirstlane_b32 s52, v3
-; SI-NEXT: v_readfirstlane_b32 s55, v2
+; SI-NEXT: v_readfirstlane_b32 s61, v18
+; SI-NEXT: v_readfirstlane_b32 s60, v17
+; SI-NEXT: v_readfirstlane_b32 s15, v16
+; SI-NEXT: v_readfirstlane_b32 s14, v15
+; SI-NEXT: v_readfirstlane_b32 s59, v14
+; SI-NEXT: v_readfirstlane_b32 s58, v13
+; SI-NEXT: v_readfirstlane_b32 s41, v12
+; SI-NEXT: v_readfirstlane_b32 s40, v11
+; SI-NEXT: v_readfirstlane_b32 s47, v10
+; SI-NEXT: v_readfirstlane_b32 s46, v9
+; SI-NEXT: v_readfirstlane_b32 s45, v8
+; SI-NEXT: v_readfirstlane_b32 s44, v7
+; SI-NEXT: v_readfirstlane_b32 s43, v6
+; SI-NEXT: v_readfirstlane_b32 s42, v5
+; SI-NEXT: v_readfirstlane_b32 s63, v4
+; SI-NEXT: v_readfirstlane_b32 s62, v3
+; SI-NEXT: v_readfirstlane_b32 s95, v2
; SI-NEXT: s_cmp_lg_u32 s4, 0
-; SI-NEXT: v_readfirstlane_b32 s54, v1
+; SI-NEXT: v_readfirstlane_b32 s94, v1
; SI-NEXT: ; implicit-def: $vgpr61 : SGPR spill to VGPR lane
; SI-NEXT: ; implicit-def: $vgpr62 : SGPR spill to VGPR lane
; SI-NEXT: s_cbranch_scc0 .LBB37_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_lshr_b32 s4, s53, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 18
-; SI-NEXT: s_lshr_b32 s4, s53, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 17
-; SI-NEXT: s_lshr_b32 s4, s55, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 16
-; SI-NEXT: s_lshr_b32 s4, s55, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 15
-; SI-NEXT: s_lshr_b32 s4, s55, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 14
-; SI-NEXT: s_lshr_b32 s4, s29, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 13
-; SI-NEXT: s_lshr_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 12
-; SI-NEXT: s_lshr_b32 s4, s29, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 11
-; SI-NEXT: s_lshr_b32 s4, s27, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 10
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 9
-; SI-NEXT: s_lshr_b32 s4, s27, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 8
-; SI-NEXT: s_lshr_b32 s4, s25, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 7
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 6
-; SI-NEXT: s_lshr_b32 s4, s25, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 5
-; SI-NEXT: s_lshr_b32 s4, s23, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 4
-; SI-NEXT: s_lshr_b32 s4, s23, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 3
-; SI-NEXT: s_lshr_b32 s4, s23, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 2
-; SI-NEXT: s_lshr_b32 s4, s21, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 1
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 0
-; SI-NEXT: s_lshr_b32 s4, s21, 8
-; SI-NEXT: v_writelane_b32 v61, s4, 63
-; SI-NEXT: s_lshr_b32 s4, s19, 24
-; SI-NEXT: v_writelane_b32 v61, s4, 62
-; SI-NEXT: s_lshr_b32 s4, s19, 16
-; SI-NEXT: v_writelane_b32 v61, s4, 61
-; SI-NEXT: s_lshr_b32 s4, s19, 8
-; SI-NEXT: v_writelane_b32 v61, s4, 60
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: v_writelane_b32 v61, s4, 59
-; SI-NEXT: s_lshr_b32 s4, s17, 8
-; SI-NEXT: v_writelane_b32 v61, s4, 58
-; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 24
-; SI-NEXT: v_writelane_b32 v61, s4, 56
-; SI-NEXT: v_writelane_b32 v61, s5, 57
-; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 16
-; SI-NEXT: v_writelane_b32 v61, s4, 54
-; SI-NEXT: v_writelane_b32 v61, s5, 55
-; SI-NEXT: s_lshr_b64 s[12:13], s[30:31], 24
-; SI-NEXT: v_writelane_b32 v61, s12, 52
-; SI-NEXT: v_writelane_b32 v61, s13, 53
-; SI-NEXT: s_lshr_b64 s[40:41], s[30:31], 8
-; SI-NEXT: v_writelane_b32 v61, s40, 50
-; SI-NEXT: v_writelane_b32 v61, s41, 51
-; SI-NEXT: s_lshr_b64 s[40:41], s[34:35], 24
-; SI-NEXT: v_writelane_b32 v61, s40, 48
-; SI-NEXT: v_writelane_b32 v61, s41, 49
-; SI-NEXT: s_lshr_b64 s[40:41], s[34:35], 16
-; SI-NEXT: v_writelane_b32 v61, s40, 46
-; SI-NEXT: v_writelane_b32 v61, s41, 47
-; SI-NEXT: s_lshr_b64 s[40:41], s[34:35], 8
-; SI-NEXT: v_writelane_b32 v61, s40, 44
-; SI-NEXT: v_writelane_b32 v61, s41, 45
-; SI-NEXT: s_lshr_b64 s[40:41], s[36:37], 24
-; SI-NEXT: v_writelane_b32 v61, s40, 42
-; SI-NEXT: v_writelane_b32 v61, s41, 43
-; SI-NEXT: s_lshr_b64 s[40:41], s[36:37], 16
-; SI-NEXT: v_writelane_b32 v61, s40, 40
-; SI-NEXT: v_writelane_b32 v61, s41, 41
-; SI-NEXT: s_lshr_b64 s[40:41], s[36:37], 8
-; SI-NEXT: v_writelane_b32 v61, s40, 38
-; SI-NEXT: v_writelane_b32 v61, s41, 39
-; SI-NEXT: s_lshr_b64 s[40:41], s[38:39], 24
-; SI-NEXT: v_writelane_b32 v61, s40, 36
-; SI-NEXT: v_writelane_b32 v61, s41, 37
-; SI-NEXT: s_lshr_b64 s[42:43], s[50:51], 24
+; SI-NEXT: s_lshr_b64 s[6:7], s[60:61], 8
+; SI-NEXT: v_writelane_b32 v61, s6, 62
+; SI-NEXT: v_writelane_b32 v61, s7, 63
+; SI-NEXT: s_lshr_b64 s[6:7], s[14:15], 24
+; SI-NEXT: v_writelane_b32 v61, s6, 60
+; SI-NEXT: v_writelane_b32 v61, s7, 61
+; SI-NEXT: s_lshr_b64 s[6:7], s[14:15], 16
+; SI-NEXT: v_writelane_b32 v61, s6, 58
+; SI-NEXT: v_writelane_b32 v61, s7, 59
+; SI-NEXT: s_lshr_b64 s[8:9], s[58:59], 24
+; SI-NEXT: v_writelane_b32 v61, s8, 56
+; SI-NEXT: v_writelane_b32 v61, s9, 57
+; SI-NEXT: s_lshr_b64 s[8:9], s[58:59], 16
+; SI-NEXT: v_writelane_b32 v61, s8, 54
+; SI-NEXT: v_writelane_b32 v61, s9, 55
+; SI-NEXT: s_lshr_b64 s[10:11], s[58:59], 8
+; SI-NEXT: v_writelane_b32 v61, s10, 52
+; SI-NEXT: v_writelane_b32 v61, s11, 53
+; SI-NEXT: s_lshr_b64 s[10:11], s[40:41], 24
+; SI-NEXT: v_writelane_b32 v61, s10, 50
+; SI-NEXT: v_writelane_b32 v61, s11, 51
+; SI-NEXT: s_lshr_b64 s[10:11], s[40:41], 16
+; SI-NEXT: v_writelane_b32 v61, s10, 48
+; SI-NEXT: v_writelane_b32 v61, s11, 49
+; SI-NEXT: s_lshr_b64 s[12:13], s[40:41], 8
+; SI-NEXT: v_writelane_b32 v61, s12, 46
+; SI-NEXT: v_writelane_b32 v61, s13, 47
+; SI-NEXT: s_lshr_b64 s[12:13], s[46:47], 24
+; SI-NEXT: v_writelane_b32 v61, s12, 44
+; SI-NEXT: s_lshr_b32 s48, s59, 24
+; SI-NEXT: s_lshr_b32 s50, s59, 16
+; SI-NEXT: s_lshr_b32 s52, s59, 8
+; SI-NEXT: s_lshr_b32 s37, s41, 24
+; SI-NEXT: s_lshr_b32 s39, s41, 16
+; SI-NEXT: s_lshr_b32 s49, s41, 8
+; SI-NEXT: s_lshr_b32 s99, s47, 24
+; SI-NEXT: s_lshr_b32 s36, s47, 16
+; SI-NEXT: s_lshr_b32 s38, s47, 8
+; SI-NEXT: s_mov_b64 s[8:9], s[58:59]
+; SI-NEXT: s_mov_b64 s[10:11], s[40:41]
+; SI-NEXT: v_writelane_b32 v61, s13, 45
+; SI-NEXT: s_lshr_b64 s[40:41], s[46:47], 16
+; SI-NEXT: s_mov_b64 s[12:13], s[46:47]
+; SI-NEXT: s_lshr_b64 s[58:59], s[46:47], 8
+; SI-NEXT: s_lshr_b64 s[46:47], s[42:43], 24
+; SI-NEXT: v_writelane_b32 v61, s46, 42
+; SI-NEXT: v_writelane_b32 v61, s47, 43
+; SI-NEXT: s_lshr_b64 s[46:47], s[42:43], 16
+; SI-NEXT: v_writelane_b32 v61, s46, 40
+; SI-NEXT: s_lshr_b32 s85, s43, 24
+; SI-NEXT: s_lshr_b32 s87, s43, 16
+; SI-NEXT: s_lshr_b32 s97, s43, 8
+; SI-NEXT: v_writelane_b32 v61, s47, 41
+; SI-NEXT: s_mov_b64 s[74:75], s[42:43]
+; SI-NEXT: s_lshr_b64 s[42:43], s[42:43], 8
+; SI-NEXT: v_writelane_b32 v61, s42, 38
+; SI-NEXT: v_writelane_b32 v61, s43, 39
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 24
+; SI-NEXT: v_writelane_b32 v61, s42, 36
+; SI-NEXT: v_writelane_b32 v61, s43, 37
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 16
; SI-NEXT: v_writelane_b32 v61, s42, 34
; SI-NEXT: v_writelane_b32 v61, s43, 35
-; SI-NEXT: s_lshr_b64 s[42:43], s[50:51], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 8
; SI-NEXT: v_writelane_b32 v61, s42, 32
; SI-NEXT: v_writelane_b32 v61, s43, 33
-; SI-NEXT: s_lshr_b64 s[42:43], s[50:51], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[94:95], 24
; SI-NEXT: v_writelane_b32 v61, s42, 30
; SI-NEXT: v_writelane_b32 v61, s43, 31
-; SI-NEXT: s_lshr_b64 s[42:43], s[52:53], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[94:95], 16
; SI-NEXT: v_writelane_b32 v61, s42, 28
; SI-NEXT: v_writelane_b32 v61, s43, 29
-; SI-NEXT: s_lshr_b64 s[42:43], s[52:53], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[94:95], 8
; SI-NEXT: v_writelane_b32 v61, s42, 26
; SI-NEXT: v_writelane_b32 v61, s43, 27
-; SI-NEXT: s_lshr_b64 s[42:43], s[52:53], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 24
; SI-NEXT: v_writelane_b32 v61, s42, 24
; SI-NEXT: v_writelane_b32 v61, s43, 25
-; SI-NEXT: s_lshr_b64 s[42:43], s[54:55], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 16
; SI-NEXT: v_writelane_b32 v61, s42, 22
; SI-NEXT: v_writelane_b32 v61, s43, 23
-; SI-NEXT: s_lshr_b64 s[42:43], s[54:55], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 8
; SI-NEXT: v_writelane_b32 v61, s42, 20
; SI-NEXT: v_writelane_b32 v61, s43, 21
-; SI-NEXT: s_lshr_b64 s[42:43], s[54:55], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 24
; SI-NEXT: v_writelane_b32 v61, s42, 18
+; SI-NEXT: s_lshr_b32 s4, s27, 24
; SI-NEXT: v_writelane_b32 v61, s43, 19
-; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 16
+; SI-NEXT: v_writelane_b32 v62, s4, 17
+; SI-NEXT: s_lshr_b32 s4, s25, 24
; SI-NEXT: v_writelane_b32 v61, s42, 16
+; SI-NEXT: v_writelane_b32 v62, s4, 16
+; SI-NEXT: s_lshr_b32 s4, s25, 16
; SI-NEXT: v_writelane_b32 v61, s43, 17
-; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 8
+; SI-NEXT: v_writelane_b32 v62, s4, 15
+; SI-NEXT: s_lshr_b32 s4, s23, 24
; SI-NEXT: v_writelane_b32 v61, s42, 14
+; SI-NEXT: v_writelane_b32 v62, s4, 14
+; SI-NEXT: s_lshr_b32 s4, s23, 16
; SI-NEXT: v_writelane_b32 v61, s43, 15
-; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 24
+; SI-NEXT: v_writelane_b32 v62, s4, 13
+; SI-NEXT: s_lshr_b32 s4, s23, 8
; SI-NEXT: v_writelane_b32 v61, s42, 12
+; SI-NEXT: v_writelane_b32 v62, s4, 12
+; SI-NEXT: s_lshr_b32 s4, s21, 24
; SI-NEXT: v_writelane_b32 v61, s43, 13
-; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 16
+; SI-NEXT: v_writelane_b32 v62, s4, 11
+; SI-NEXT: s_lshr_b32 s4, s21, 16
; SI-NEXT: v_writelane_b32 v61, s42, 10
+; SI-NEXT: v_writelane_b32 v62, s4, 10
+; SI-NEXT: s_lshr_b32 s4, s21, 8
; SI-NEXT: v_writelane_b32 v61, s43, 11
-; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 8
+; SI-NEXT: v_writelane_b32 v62, s4, 9
+; SI-NEXT: s_lshr_b32 s4, s19, 24
; SI-NEXT: v_writelane_b32 v61, s42, 8
+; SI-NEXT: v_writelane_b32 v62, s4, 8
+; SI-NEXT: s_lshr_b32 s4, s19, 16
; SI-NEXT: v_writelane_b32 v61, s43, 9
-; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[22:23], 24
+; SI-NEXT: v_writelane_b32 v62, s4, 7
+; SI-NEXT: s_lshr_b32 s4, s19, 8
; SI-NEXT: v_writelane_b32 v61, s42, 6
+; SI-NEXT: v_writelane_b32 v62, s4, 6
+; SI-NEXT: s_lshr_b32 s4, s17, 16
; SI-NEXT: v_writelane_b32 v61, s43, 7
-; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[22:23], 16
+; SI-NEXT: v_writelane_b32 v62, s4, 5
+; SI-NEXT: s_lshr_b32 s4, s17, 8
; SI-NEXT: v_writelane_b32 v61, s42, 4
+; SI-NEXT: v_writelane_b32 v62, s4, 4
+; SI-NEXT: s_lshr_b64 s[4:5], s[60:61], 24
; SI-NEXT: v_writelane_b32 v61, s43, 5
-; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[22:23], 8
+; SI-NEXT: v_writelane_b32 v62, s4, 2
; SI-NEXT: v_writelane_b32 v61, s42, 2
+; SI-NEXT: v_writelane_b32 v62, s5, 3
+; SI-NEXT: s_lshr_b64 s[4:5], s[60:61], 16
; SI-NEXT: v_writelane_b32 v61, s43, 3
-; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[20:21], 24
+; SI-NEXT: v_writelane_b32 v62, s4, 0
; SI-NEXT: v_writelane_b32 v61, s42, 0
-; SI-NEXT: s_lshr_b32 s15, s9, 24
-; SI-NEXT: s_lshr_b32 s69, s9, 16
-; SI-NEXT: s_lshr_b32 s70, s9, 8
-; SI-NEXT: s_lshr_b32 s10, s31, 24
-; SI-NEXT: s_lshr_b32 s14, s31, 16
-; SI-NEXT: s_lshr_b32 s68, s31, 8
-; SI-NEXT: s_lshr_b32 s98, s35, 24
-; SI-NEXT: s_lshr_b32 s7, s35, 16
-; SI-NEXT: s_lshr_b32 s11, s35, 8
-; SI-NEXT: s_lshr_b32 s96, s37, 24
-; SI-NEXT: s_lshr_b32 s67, s37, 16
-; SI-NEXT: s_lshr_b32 s99, s37, 8
-; SI-NEXT: s_lshr_b32 s85, s39, 24
-; SI-NEXT: s_lshr_b32 s87, s39, 16
-; SI-NEXT: s_lshr_b32 s97, s39, 8
-; SI-NEXT: s_lshr_b32 s82, s49, 24
-; SI-NEXT: s_lshr_b32 s84, s49, 16
-; SI-NEXT: s_lshr_b32 s86, s49, 8
-; SI-NEXT: s_lshr_b32 s71, s51, 24
-; SI-NEXT: s_lshr_b32 s81, s51, 16
-; SI-NEXT: s_lshr_b32 s83, s51, 8
-; SI-NEXT: s_lshr_b32 s80, s53, 8
-; SI-NEXT: s_lshr_b32 s66, s17, 24
-; SI-NEXT: s_mov_b64 s[4:5], s[8:9]
-; SI-NEXT: s_lshr_b64 s[8:9], s[8:9], 8
-; SI-NEXT: s_lshr_b64 s[12:13], s[30:31], 16
-; SI-NEXT: s_lshr_b64 s[40:41], s[38:39], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[38:39], 8
-; SI-NEXT: s_lshr_b64 s[44:45], s[48:49], 24
-; SI-NEXT: s_lshr_b64 s[56:57], s[48:49], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[48:49], 8
+; SI-NEXT: s_lshr_b32 s54, s61, 24
+; SI-NEXT: s_lshr_b32 s64, s61, 16
+; SI-NEXT: s_lshr_b32 s65, s61, 8
+; SI-NEXT: s_lshr_b32 s51, s15, 24
+; SI-NEXT: s_lshr_b32 s53, s15, 16
+; SI-NEXT: s_lshr_b32 s55, s15, 8
+; SI-NEXT: s_lshr_b32 s96, s45, 24
+; SI-NEXT: s_lshr_b32 s98, s45, 16
+; SI-NEXT: s_lshr_b32 s35, s45, 8
+; SI-NEXT: s_lshr_b32 s82, s63, 24
+; SI-NEXT: s_lshr_b32 s84, s63, 16
+; SI-NEXT: s_lshr_b32 s86, s63, 8
+; SI-NEXT: s_lshr_b32 s71, s95, 24
+; SI-NEXT: s_lshr_b32 s81, s95, 16
+; SI-NEXT: s_lshr_b32 s83, s95, 8
+; SI-NEXT: s_lshr_b32 s68, s29, 24
+; SI-NEXT: s_lshr_b32 s70, s29, 16
+; SI-NEXT: s_lshr_b32 s80, s29, 8
+; SI-NEXT: s_lshr_b32 s67, s27, 16
+; SI-NEXT: s_lshr_b32 s69, s27, 8
+; SI-NEXT: s_lshr_b32 s66, s25, 8
+; SI-NEXT: s_lshr_b32 s34, s17, 24
+; SI-NEXT: v_writelane_b32 v62, s5, 1
+; SI-NEXT: s_mov_b64 s[4:5], s[60:61]
+; SI-NEXT: s_mov_b64 s[6:7], s[14:15]
+; SI-NEXT: s_lshr_b64 s[14:15], s[14:15], 8
+; SI-NEXT: s_lshr_b64 s[60:61], s[44:45], 24
+; SI-NEXT: s_lshr_b64 s[72:73], s[44:45], 16
+; SI-NEXT: s_mov_b64 s[56:57], s[44:45]
+; SI-NEXT: s_lshr_b64 s[44:45], s[44:45], 8
; SI-NEXT: v_writelane_b32 v61, s43, 1
-; SI-NEXT: s_lshr_b64 s[74:75], s[22:23], 24
-; SI-NEXT: s_lshr_b64 s[46:47], s[22:23], 16
-; SI-NEXT: s_lshr_b64 s[76:77], s[22:23], 8
-; SI-NEXT: s_lshr_b64 s[58:59], s[20:21], 24
-; SI-NEXT: s_lshr_b64 s[60:61], s[20:21], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[20:21], 8
-; SI-NEXT: s_lshr_b64 s[88:89], s[18:19], 24
-; SI-NEXT: s_lshr_b64 s[90:91], s[18:19], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[18:19], 8
-; SI-NEXT: s_lshr_b64 s[94:95], s[16:17], 24
+; SI-NEXT: s_lshr_b64 s[46:47], s[20:21], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[20:21], 8
+; SI-NEXT: s_lshr_b64 s[78:79], s[18:19], 24
+; SI-NEXT: s_lshr_b64 s[88:89], s[18:19], 16
+; SI-NEXT: s_lshr_b64 s[90:91], s[18:19], 8
+; SI-NEXT: s_lshr_b64 s[92:93], s[16:17], 24
; SI-NEXT: s_lshr_b64 vcc, s[16:17], 16
-; SI-NEXT: s_lshr_b64 s[42:43], s[16:17], 8
-; SI-NEXT: s_mov_b64 s[64:65], 0
+; SI-NEXT: s_lshr_b64 s[76:77], s[16:17], 8
+; SI-NEXT: s_mov_b64 s[30:31], 0
; SI-NEXT: s_branch .LBB37_3
; SI-NEXT: .LBB37_2:
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr67
+; SI-NEXT: s_mov_b64 s[6:7], s[14:15]
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: s_mov_b64 s[74:75], s[42:43]
+; SI-NEXT: v_writelane_b32 v61, s14, 0
+; SI-NEXT: v_writelane_b32 v61, s15, 1
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[56:57], s[44:45]
+; SI-NEXT: v_writelane_b32 v61, s14, 2
+; SI-NEXT: v_writelane_b32 v61, s15, 3
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[12:13], s[46:47]
+; SI-NEXT: v_writelane_b32 v61, s14, 4
+; SI-NEXT: v_writelane_b32 v61, s15, 5
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[10:11], s[40:41]
+; SI-NEXT: v_writelane_b32 v61, s14, 6
+; SI-NEXT: v_writelane_b32 v61, s15, 7
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[8:9], s[58:59]
+; SI-NEXT: v_writelane_b32 v61, s14, 8
+; SI-NEXT: v_writelane_b32 v61, s15, 9
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[4:5], s[60:61]
+; SI-NEXT: v_writelane_b32 v61, s14, 10
+; SI-NEXT: v_writelane_b32 v61, s15, 11
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[30:31], -1
+; SI-NEXT: v_writelane_b32 v61, s14, 12
+; SI-NEXT: v_writelane_b32 v61, s15, 13
+; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr66
-; SI-NEXT: s_mov_b64 s[4:5], s[8:9]
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b64 s[64:65], -1
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr83
; SI-NEXT: ; implicit-def: $sgpr81
; SI-NEXT: ; implicit-def: $sgpr71
@@ -43662,289 +43722,240 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: ; implicit-def: $sgpr97
; SI-NEXT: ; implicit-def: $sgpr87
; SI-NEXT: ; implicit-def: $sgpr85
-; SI-NEXT: ; implicit-def: $sgpr99
-; SI-NEXT: ; implicit-def: $sgpr96
-; SI-NEXT: ; implicit-def: $sgpr11
; SI-NEXT: ; implicit-def: $sgpr98
-; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr14
-; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr69
-; SI-NEXT: ; implicit-def: $sgpr15
-; SI-NEXT: ; implicit-def: $sgpr42
+; SI-NEXT: ; implicit-def: $sgpr96
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr99
+; SI-NEXT: ; implicit-def: $sgpr49
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr50
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr53
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr92
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr56
; SI-NEXT: ; implicit-def: $sgpr44
; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: ; implicit-def: $sgpr60
+; SI-NEXT: ; implicit-def: $sgpr58
; SI-NEXT: ; implicit-def: $sgpr40
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 0
-; SI-NEXT: v_writelane_b32 v61, s7, 1
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 2
-; SI-NEXT: v_writelane_b32 v61, s7, 3
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 4
-; SI-NEXT: v_writelane_b32 v61, s7, 5
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 6
-; SI-NEXT: v_writelane_b32 v61, s7, 7
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 8
-; SI-NEXT: v_writelane_b32 v61, s7, 9
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 10
-; SI-NEXT: v_writelane_b32 v61, s7, 11
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 12
-; SI-NEXT: v_writelane_b32 v61, s7, 13
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 14
-; SI-NEXT: v_writelane_b32 v61, s7, 15
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 16
-; SI-NEXT: v_writelane_b32 v61, s7, 17
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 18
-; SI-NEXT: v_writelane_b32 v61, s7, 19
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 20
-; SI-NEXT: v_writelane_b32 v61, s7, 21
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 22
-; SI-NEXT: v_writelane_b32 v61, s7, 23
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 24
-; SI-NEXT: v_writelane_b32 v61, s7, 25
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 26
-; SI-NEXT: v_writelane_b32 v61, s7, 27
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 28
-; SI-NEXT: v_writelane_b32 v61, s7, 29
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 30
-; SI-NEXT: v_writelane_b32 v61, s7, 31
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 32
-; SI-NEXT: v_writelane_b32 v61, s7, 33
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 34
-; SI-NEXT: v_writelane_b32 v61, s7, 35
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 36
-; SI-NEXT: v_writelane_b32 v61, s7, 37
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 38
-; SI-NEXT: v_writelane_b32 v61, s7, 39
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 40
-; SI-NEXT: v_writelane_b32 v61, s7, 41
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 42
-; SI-NEXT: v_writelane_b32 v61, s7, 43
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 44
-; SI-NEXT: v_writelane_b32 v61, s7, 45
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 46
-; SI-NEXT: v_writelane_b32 v61, s7, 47
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 48
-; SI-NEXT: v_writelane_b32 v61, s7, 49
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 50
-; SI-NEXT: v_writelane_b32 v61, s7, 51
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 52
-; SI-NEXT: v_writelane_b32 v61, s7, 53
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 54
-; SI-NEXT: v_writelane_b32 v61, s7, 55
-; SI-NEXT: v_writelane_b32 v61, s66, 56
-; SI-NEXT: v_writelane_b32 v61, s67, 57
-; SI-NEXT: ; implicit-def: $sgpr66
+; SI-NEXT: v_writelane_b32 v61, s14, 14
+; SI-NEXT: v_writelane_b32 v61, s15, 15
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 16
+; SI-NEXT: v_writelane_b32 v61, s15, 17
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 18
+; SI-NEXT: v_writelane_b32 v61, s15, 19
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 20
+; SI-NEXT: v_writelane_b32 v61, s15, 21
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 22
+; SI-NEXT: v_writelane_b32 v61, s15, 23
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 24
+; SI-NEXT: v_writelane_b32 v61, s15, 25
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 26
+; SI-NEXT: v_writelane_b32 v61, s15, 27
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 28
+; SI-NEXT: v_writelane_b32 v61, s15, 29
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 30
+; SI-NEXT: v_writelane_b32 v61, s15, 31
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 32
+; SI-NEXT: v_writelane_b32 v61, s15, 33
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 34
+; SI-NEXT: v_writelane_b32 v61, s15, 35
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 36
+; SI-NEXT: v_writelane_b32 v61, s15, 37
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 38
+; SI-NEXT: v_writelane_b32 v61, s15, 39
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 40
+; SI-NEXT: v_writelane_b32 v61, s15, 41
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 42
+; SI-NEXT: v_writelane_b32 v61, s15, 43
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 44
+; SI-NEXT: v_writelane_b32 v61, s15, 45
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 46
+; SI-NEXT: v_writelane_b32 v61, s15, 47
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 48
+; SI-NEXT: v_writelane_b32 v61, s15, 49
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 50
+; SI-NEXT: v_writelane_b32 v61, s15, 51
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 52
+; SI-NEXT: v_writelane_b32 v61, s15, 53
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 54
+; SI-NEXT: v_writelane_b32 v61, s15, 55
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 56
+; SI-NEXT: v_writelane_b32 v61, s15, 57
+; SI-NEXT: v_writelane_b32 v61, s34, 58
+; SI-NEXT: v_writelane_b32 v61, s35, 59
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s34, 60
+; SI-NEXT: v_writelane_b32 v61, s35, 61
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: v_writelane_b32 v61, s34, 62
+; SI-NEXT: v_writelane_b32 v61, s35, 63
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: v_writelane_b32 v62, s34, 0
+; SI-NEXT: v_writelane_b32 v62, s35, 1
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: v_writelane_b32 v62, s34, 2
+; SI-NEXT: v_writelane_b32 v62, s35, 3
+; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: .LBB37_3: ; %Flow
-; SI-NEXT: s_and_b64 s[64:65], s[64:65], exec
-; SI-NEXT: s_cselect_b32 s6, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s6, 1
+; SI-NEXT: s_and_b64 s[30:31], s[30:31], exec
+; SI-NEXT: s_cselect_b32 s15, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s15, 1
; SI-NEXT: s_cbranch_scc1 .LBB37_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_add_f32_e64 v1, s38, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s12, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s44, v1
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_add_f32_e64 v1, s36, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s10, 1.0
; SI-NEXT: v_add_f32_e64 v22, s26, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s26, v1
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_add_f32_e64 v1, s34, 1.0
-; SI-NEXT: v_add_f32_e64 v29, s20, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s8, 1.0
+; SI-NEXT: v_add_f32_e64 v28, s20, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s20, v1
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_add_f32_e64 v1, s30, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s6, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s14, v1
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_f32_e64 v1, s5, 1.0
; SI-NEXT: v_add_f32_e64 v3, s4, 1.0
-; SI-NEXT: v_add_f32_e64 v15, s53, 1.0
+; SI-NEXT: v_add_f32_e64 v16, s95, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s8, v3
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v1
-; SI-NEXT: v_add_f32_e64 v17, s55, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v15
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v16
+; SI-NEXT: v_add_f32_e64 v18, s29, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v17
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v16
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v17
-; SI-NEXT: v_add_f32_e64 v19, s29, 1.0
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v18
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v17
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v18
+; SI-NEXT: v_add_f32_e64 v21, s27, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v19
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v18
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v19
-; SI-NEXT: v_add_f32_e64 v21, s27, 1.0
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v21
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v19
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v21
+; SI-NEXT: v_add_f32_e64 v23, s25, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v21
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v21
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v21
-; SI-NEXT: v_add_f32_e64 v23, s25, 1.0
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v23
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v21
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v23
+; SI-NEXT: v_add_f32_e64 v25, s23, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v23
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v23
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v25
; SI-NEXT: v_add_f32_e64 v34, s17, 1.0
-; SI-NEXT: v_add_f32_e64 v38, s16, 1.0
-; SI-NEXT: v_add_f32_e64 v31, s19, 1.0
+; SI-NEXT: v_add_f32_e64 v36, s16, 1.0
+; SI-NEXT: v_add_f32_e64 v29, s19, 1.0
; SI-NEXT: v_add_f32_e64 v33, s18, 1.0
; SI-NEXT: v_add_f32_e64 v27, s21, 1.0
-; SI-NEXT: v_add_f32_e64 v25, s23, 1.0
; SI-NEXT: v_add_f32_e64 v26, s22, 1.0
; SI-NEXT: v_add_f32_e64 v24, s24, 1.0
; SI-NEXT: v_add_f32_e64 v20, s28, 1.0
-; SI-NEXT: v_add_f32_e64 v18, s54, 1.0
-; SI-NEXT: v_add_f32_e64 v16, s52, 1.0
-; SI-NEXT: v_add_f32_e64 v13, s51, 1.0
-; SI-NEXT: v_add_f32_e64 v14, s50, 1.0
-; SI-NEXT: v_add_f32_e64 v11, s49, 1.0
-; SI-NEXT: v_add_f32_e64 v12, s48, 1.0
-; SI-NEXT: v_add_f32_e64 v9, s39, 1.0
-; SI-NEXT: v_add_f32_e64 v7, s37, 1.0
-; SI-NEXT: v_add_f32_e64 v5, s35, 1.0
-; SI-NEXT: v_add_f32_e64 v2, s31, 1.0
+; SI-NEXT: v_add_f32_e64 v19, s94, 1.0
+; SI-NEXT: v_add_f32_e64 v14, s63, 1.0
+; SI-NEXT: v_add_f32_e64 v17, s62, 1.0
+; SI-NEXT: v_add_f32_e64 v12, s75, 1.0
+; SI-NEXT: v_add_f32_e64 v15, s74, 1.0
+; SI-NEXT: v_add_f32_e64 v10, s57, 1.0
+; SI-NEXT: v_add_f32_e64 v13, s56, 1.0
+; SI-NEXT: v_add_f32_e64 v8, s13, 1.0
+; SI-NEXT: v_add_f32_e64 v6, s11, 1.0
+; SI-NEXT: v_add_f32_e64 v4, s9, 1.0
+; SI-NEXT: v_add_f32_e64 v2, s7, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v23
-; SI-NEXT: v_readfirstlane_b32 s98, v38
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v25
+; SI-NEXT: v_readfirstlane_b32 s98, v36
; SI-NEXT: v_readfirstlane_b32 s99, v34
; SI-NEXT: v_readfirstlane_b32 s84, v33
-; SI-NEXT: v_readfirstlane_b32 s85, v31
-; SI-NEXT: v_readfirstlane_b32 s70, v29
+; SI-NEXT: v_readfirstlane_b32 s85, v29
+; SI-NEXT: v_readfirstlane_b32 s70, v28
; SI-NEXT: v_readfirstlane_b32 s71, v27
; SI-NEXT: v_readfirstlane_b32 s64, v26
; SI-NEXT: v_readfirstlane_b32 s65, v25
-; SI-NEXT: v_readfirstlane_b32 s92, v24
-; SI-NEXT: v_readfirstlane_b32 s93, v23
-; SI-NEXT: v_readfirstlane_b32 s78, v22
-; SI-NEXT: v_readfirstlane_b32 s79, v21
-; SI-NEXT: v_readfirstlane_b32 s76, v20
-; SI-NEXT: v_readfirstlane_b32 s77, v19
-; SI-NEXT: v_readfirstlane_b32 s88, v18
-; SI-NEXT: v_readfirstlane_b32 s89, v17
-; SI-NEXT: v_readfirstlane_b32 s90, v16
-; SI-NEXT: v_readfirstlane_b32 s91, v15
-; SI-NEXT: v_readfirstlane_b32 s58, v14
-; SI-NEXT: v_readfirstlane_b32 s59, v13
-; SI-NEXT: v_readfirstlane_b32 s42, v12
-; SI-NEXT: v_readfirstlane_b32 s43, v11
-; SI-NEXT: v_readfirstlane_b32 s45, v9
-; SI-NEXT: v_readfirstlane_b32 s27, v7
-; SI-NEXT: v_readfirstlane_b32 s21, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v24
+; SI-NEXT: v_readfirstlane_b32 s51, v23
+; SI-NEXT: v_readfirstlane_b32 s36, v22
+; SI-NEXT: v_readfirstlane_b32 s37, v21
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v20
+; SI-NEXT: v_readfirstlane_b32 vcc_hi, v18
+; SI-NEXT: v_readfirstlane_b32 s88, v19
+; SI-NEXT: v_readfirstlane_b32 s89, v16
+; SI-NEXT: v_readfirstlane_b32 s76, v17
+; SI-NEXT: v_readfirstlane_b32 s77, v14
+; SI-NEXT: v_readfirstlane_b32 s62, v15
+; SI-NEXT: v_readfirstlane_b32 s63, v12
+; SI-NEXT: v_readfirstlane_b32 s42, v13
+; SI-NEXT: v_readfirstlane_b32 s43, v10
+; SI-NEXT: v_readfirstlane_b32 s45, v8
+; SI-NEXT: v_readfirstlane_b32 s27, v6
+; SI-NEXT: v_readfirstlane_b32 s21, v4
; SI-NEXT: v_readfirstlane_b32 s15, v2
; SI-NEXT: v_readfirstlane_b32 s9, v1
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v25
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v25
; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 24
; SI-NEXT: s_lshr_b64 s[6:7], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[8:9], s[8:9], 8
@@ -43959,30 +43970,30 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: s_lshr_b64 s[26:27], s[26:27], 8
; SI-NEXT: s_lshr_b64 s[28:29], s[44:45], 24
; SI-NEXT: s_lshr_b64 s[40:41], s[44:45], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[44:45], 8
-; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 24
-; SI-NEXT: s_lshr_b64 s[56:57], s[42:43], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[42:43], 8
-; SI-NEXT: s_lshr_b64 s[42:43], s[58:59], 24
-; SI-NEXT: s_lshr_b64 s[46:47], s[58:59], 16
-; SI-NEXT: s_lshr_b64 s[58:59], s[58:59], 8
-; SI-NEXT: s_lshr_b64 s[60:61], s[90:91], 24
-; SI-NEXT: s_lshr_b64 s[74:75], s[90:91], 16
-; SI-NEXT: s_lshr_b64 s[30:31], s[90:91], 8
-; SI-NEXT: s_lshr_b64 s[34:35], s[88:89], 24
-; SI-NEXT: s_lshr_b64 s[36:37], s[88:89], 16
-; SI-NEXT: s_lshr_b64 s[38:39], s[88:89], 8
-; SI-NEXT: s_lshr_b64 s[48:49], s[76:77], 24
-; SI-NEXT: s_lshr_b64 s[50:51], s[76:77], 16
-; SI-NEXT: s_lshr_b64 s[52:53], s[76:77], 8
-; SI-NEXT: s_lshr_b64 s[54:55], s[78:79], 24
-; SI-NEXT: s_lshr_b64 s[76:77], s[78:79], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 8
-; SI-NEXT: s_lshr_b64 s[88:89], s[92:93], 24
-; SI-NEXT: s_lshr_b64 s[90:91], s[92:93], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[92:93], 8
-; SI-NEXT: s_lshr_b64 s[94:95], s[64:65], 24
-; SI-NEXT: s_lshr_b64 vcc, s[64:65], 16
+; SI-NEXT: s_lshr_b64 s[58:59], s[44:45], 8
+; SI-NEXT: s_lshr_b64 s[60:61], s[42:43], 24
+; SI-NEXT: s_lshr_b64 s[72:73], s[42:43], 16
+; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 24
+; SI-NEXT: s_lshr_b64 s[46:47], s[62:63], 16
+; SI-NEXT: s_lshr_b64 s[56:57], s[62:63], 8
+; SI-NEXT: s_lshr_b64 s[62:63], s[76:77], 24
+; SI-NEXT: s_lshr_b64 s[74:75], s[76:77], 16
+; SI-NEXT: s_lshr_b64 s[94:95], s[76:77], 8
+; SI-NEXT: s_lshr_b64 s[76:77], s[88:89], 24
+; SI-NEXT: s_lshr_b64 s[78:79], s[88:89], 16
+; SI-NEXT: s_lshr_b64 s[88:89], s[88:89], 8
+; SI-NEXT: s_lshr_b64 s[90:91], vcc, 24
+; SI-NEXT: s_lshr_b64 s[92:93], vcc, 16
+; SI-NEXT: s_lshr_b64 vcc, vcc, 8
+; SI-NEXT: s_lshr_b64 s[30:31], s[36:37], 24
+; SI-NEXT: s_lshr_b64 s[34:35], s[36:37], 16
+; SI-NEXT: s_lshr_b64 s[36:37], s[36:37], 8
+; SI-NEXT: s_lshr_b64 s[38:39], s[50:51], 24
+; SI-NEXT: s_lshr_b64 s[48:49], s[50:51], 16
+; SI-NEXT: s_lshr_b64 s[50:51], s[50:51], 8
+; SI-NEXT: s_lshr_b64 s[52:53], s[64:65], 24
+; SI-NEXT: s_lshr_b64 s[54:55], s[64:65], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 8
; SI-NEXT: s_lshr_b64 s[66:67], s[70:71], 24
; SI-NEXT: s_lshr_b64 s[68:69], s[70:71], 16
@@ -43998,353 +44009,343 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_lshrrev_b32_e32 v42, 24, v2
; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v2
; SI-NEXT: v_lshrrev_b32_e32 v44, 8, v2
-; SI-NEXT: v_lshrrev_b32_e32 v45, 24, v5
-; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v47, 8, v5
-; SI-NEXT: v_lshrrev_b32_e32 v56, 24, v7
-; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v58, 8, v7
-; SI-NEXT: v_lshrrev_b32_e32 v59, 24, v9
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v28, 8, v9
-; SI-NEXT: v_lshrrev_b32_e32 v30, 24, v11
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v35, 8, v11
-; SI-NEXT: v_lshrrev_b32_e32 v36, 24, v13
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v39, 8, v13
-; SI-NEXT: v_lshrrev_b32_e32 v48, 24, v15
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v45, 24, v4
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v4
+; SI-NEXT: v_lshrrev_b32_e32 v47, 8, v4
+; SI-NEXT: v_lshrrev_b32_e32 v56, 24, v6
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v6
+; SI-NEXT: v_lshrrev_b32_e32 v58, 8, v6
+; SI-NEXT: v_lshrrev_b32_e32 v59, 24, v8
+; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v8
+; SI-NEXT: v_lshrrev_b32_e32 v30, 8, v8
+; SI-NEXT: v_lshrrev_b32_e32 v31, 24, v10
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v10
+; SI-NEXT: v_lshrrev_b32_e32 v35, 8, v10
+; SI-NEXT: v_lshrrev_b32_e32 v37, 24, v12
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v12
+; SI-NEXT: v_lshrrev_b32_e32 v39, 8, v12
+; SI-NEXT: v_lshrrev_b32_e32 v48, 24, v14
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v14
+; SI-NEXT: v_lshrrev_b32_e32 v50, 8, v14
+; SI-NEXT: v_lshrrev_b32_e32 v51, 24, v16
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v25
-; SI-NEXT: v_lshrrev_b32_e32 v50, 8, v25
-; SI-NEXT: v_lshrrev_b32_e32 v4, 24, v27
-; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v52, 8, v27
-; SI-NEXT: v_lshrrev_b32_e32 v6, 24, v31
-; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v31
-; SI-NEXT: v_lshrrev_b32_e32 v53, 8, v31
-; SI-NEXT: v_lshrrev_b32_e32 v54, 24, v34
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v34
-; SI-NEXT: v_lshrrev_b32_e32 v10, 8, v34
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v27
+; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v27
+; SI-NEXT: v_lshrrev_b32_e32 v5, 8, v27
+; SI-NEXT: v_lshrrev_b32_e32 v7, 24, v29
+; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
+; SI-NEXT: v_lshrrev_b32_e32 v54, 8, v29
+; SI-NEXT: v_lshrrev_b32_e32 v55, 24, v34
+; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v34
+; SI-NEXT: v_lshrrev_b32_e32 v11, 8, v34
; SI-NEXT: s_branch .LBB37_6
; SI-NEXT: .LBB37_5:
; SI-NEXT: v_mov_b32_e32 v1, s16
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s38
+; SI-NEXT: v_mov_b32_e32 v1, s12
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s36
+; SI-NEXT: v_mov_b32_e32 v1, s10
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s34
+; SI-NEXT: v_mov_b32_e32 v1, s8
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s30
+; SI-NEXT: v_mov_b32_e32 v1, s6
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v1, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 58
-; SI-NEXT: v_mov_b32_e32 v10, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 59
-; SI-NEXT: v_mov_b32_e32 v55, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 60
-; SI-NEXT: v_mov_b32_e32 v53, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 61
-; SI-NEXT: v_mov_b32_e32 v8, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 62
-; SI-NEXT: v_mov_b32_e32 v6, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 63
-; SI-NEXT: v_mov_b32_e32 v52, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 0
-; SI-NEXT: v_mov_b32_e32 v51, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 1
-; SI-NEXT: v_mov_b32_e32 v4, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 2
-; SI-NEXT: v_mov_b32_e32 v50, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 3
-; SI-NEXT: v_mov_b32_e32 v3, s4
; SI-NEXT: v_readlane_b32 s4, v62, 4
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v11, s4
; SI-NEXT: v_readlane_b32 s4, v62, 5
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v9, s4
; SI-NEXT: v_readlane_b32 s4, v62, 6
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v54, s4
; SI-NEXT: v_readlane_b32 s4, v62, 7
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v53, s4
; SI-NEXT: v_readlane_b32 s4, v62, 8
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v7, s4
; SI-NEXT: v_readlane_b32 s4, v62, 9
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v5, s4
; SI-NEXT: v_readlane_b32 s4, v62, 10
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v52, s4
; SI-NEXT: v_readlane_b32 s4, v62, 11
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v3, s4
; SI-NEXT: v_readlane_b32 s4, v62, 12
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
; SI-NEXT: v_readlane_b32 s4, v62, 13
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
; SI-NEXT: v_readlane_b32 s4, v62, 14
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s66
; SI-NEXT: v_readlane_b32 s4, v62, 15
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
; SI-NEXT: v_readlane_b32 s4, v62, 16
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s69
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s67
; SI-NEXT: v_readlane_b32 s4, v62, 17
-; SI-NEXT: v_mov_b32_e32 v49, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 18
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v1, s5
-; SI-NEXT: v_mov_b32_e32 v48, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 8
-; SI-NEXT: v_readlane_b32 s5, v61, 9
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s4
+; SI-NEXT: v_readlane_b32 s4, v61, 30
+; SI-NEXT: v_readlane_b32 s5, v61, 31
; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 6
-; SI-NEXT: v_readlane_b32 s5, v61, 7
-; SI-NEXT: v_mov_b32_e32 v42, s10
+; SI-NEXT: v_readlane_b32 s4, v61, 28
+; SI-NEXT: v_readlane_b32 s5, v61, 29
+; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: v_readlane_b32 s4, v61, 26
+; SI-NEXT: v_readlane_b32 s5, v61, 27
; SI-NEXT: s_mov_b32 s10, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 4
-; SI-NEXT: v_readlane_b32 s5, v61, 5
-; SI-NEXT: v_mov_b32_e32 v43, s14
-; SI-NEXT: s_mov_b32 s14, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 2
-; SI-NEXT: v_readlane_b32 s5, v61, 3
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s80
+; SI-NEXT: v_readlane_b32 s4, v61, 24
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s80
+; SI-NEXT: v_readlane_b32 s5, v61, 25
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s70
+; SI-NEXT: s_mov_b32 s12, s4
+; SI-NEXT: v_readlane_b32 s4, v61, 22
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s68
+; SI-NEXT: v_readlane_b32 s5, v61, 23
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s83
; SI-NEXT: s_mov_b32 s16, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 0
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: v_readlane_b32 s4, v61, 20
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s15
-; SI-NEXT: v_readlane_b32 s5, v61, 1
+; SI-NEXT: v_mov_b32_e32 v13, s81
+; SI-NEXT: v_mov_b32_e32 v40, s54
+; SI-NEXT: v_readlane_b32 s5, v61, 21
; SI-NEXT: v_mov_b32_e32 v34, s17
-; SI-NEXT: v_mov_b32_e32 v31, s19
+; SI-NEXT: v_mov_b32_e32 v29, s19
; SI-NEXT: v_mov_b32_e32 v27, s21
; SI-NEXT: v_mov_b32_e32 v25, s23
; SI-NEXT: v_mov_b32_e32 v23, s25
; SI-NEXT: v_mov_b32_e32 v21, s27
-; SI-NEXT: v_mov_b32_e32 v19, s29
-; SI-NEXT: v_mov_b32_e32 v17, s55
-; SI-NEXT: v_mov_b32_e32 v15, s53
-; SI-NEXT: v_mov_b32_e32 v13, s51
-; SI-NEXT: v_mov_b32_e32 v11, s49
-; SI-NEXT: v_mov_b32_e32 v9, s39
-; SI-NEXT: v_mov_b32_e32 v7, s37
-; SI-NEXT: v_mov_b32_e32 v5, s35
-; SI-NEXT: v_mov_b32_e32 v2, s31
-; SI-NEXT: v_mov_b32_e32 v54, s66
-; SI-NEXT: v_mov_b32_e32 v35, s86
-; SI-NEXT: v_mov_b32_e32 v32, s84
-; SI-NEXT: v_mov_b32_e32 v30, s82
-; SI-NEXT: v_mov_b32_e32 v56, s96
-; SI-NEXT: v_mov_b32_e32 v47, s11
-; SI-NEXT: v_mov_b32_e32 v46, s7
-; SI-NEXT: v_mov_b32_e32 v45, s98
-; SI-NEXT: v_mov_b32_e32 v44, s68
-; SI-NEXT: v_mov_b32_e32 v41, s70
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v18, s29
+; SI-NEXT: v_mov_b32_e32 v16, s95
+; SI-NEXT: v_mov_b32_e32 v14, s63
+; SI-NEXT: v_mov_b32_e32 v12, s75
+; SI-NEXT: v_mov_b32_e32 v10, s57
+; SI-NEXT: v_mov_b32_e32 v8, s13
+; SI-NEXT: v_mov_b32_e32 v6, s11
+; SI-NEXT: v_mov_b32_e32 v4, s9
+; SI-NEXT: v_mov_b32_e32 v2, s7
+; SI-NEXT: v_mov_b32_e32 v55, s34
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v50, s86
+; SI-NEXT: v_mov_b32_e32 v49, s84
+; SI-NEXT: v_mov_b32_e32 v48, s82
+; SI-NEXT: v_mov_b32_e32 v35, s35
+; SI-NEXT: v_mov_b32_e32 v32, s98
+; SI-NEXT: v_mov_b32_e32 v31, s96
+; SI-NEXT: v_mov_b32_e32 v30, s38
+; SI-NEXT: v_mov_b32_e32 v60, s36
+; SI-NEXT: v_mov_b32_e32 v58, s49
+; SI-NEXT: v_mov_b32_e32 v57, s39
+; SI-NEXT: v_mov_b32_e32 v56, s37
+; SI-NEXT: v_mov_b32_e32 v47, s52
+; SI-NEXT: v_mov_b32_e32 v46, s50
+; SI-NEXT: v_mov_b32_e32 v45, s48
+; SI-NEXT: v_mov_b32_e32 v44, s55
+; SI-NEXT: v_mov_b32_e32 v43, s53
+; SI-NEXT: v_mov_b32_e32 v42, s51
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s48
-; SI-NEXT: v_mov_b32_e32 v14, s50
-; SI-NEXT: v_mov_b32_e32 v16, s52
-; SI-NEXT: v_mov_b32_e32 v18, s54
+; SI-NEXT: v_mov_b32_e32 v40, s64
+; SI-NEXT: v_mov_b32_e32 v41, s65
+; SI-NEXT: v_mov_b32_e32 v13, s56
+; SI-NEXT: v_mov_b32_e32 v15, s74
+; SI-NEXT: v_mov_b32_e32 v17, s62
+; SI-NEXT: v_mov_b32_e32 v19, s94
; SI-NEXT: v_mov_b32_e32 v20, s28
; SI-NEXT: v_mov_b32_e32 v22, s26
; SI-NEXT: v_mov_b32_e32 v24, s24
; SI-NEXT: v_mov_b32_e32 v26, s22
-; SI-NEXT: v_mov_b32_e32 v29, s20
+; SI-NEXT: v_mov_b32_e32 v28, s20
; SI-NEXT: v_mov_b32_e32 v33, s18
-; SI-NEXT: v_readlane_b32 s30, v61, 24
-; SI-NEXT: v_readlane_b32 s34, v61, 22
-; SI-NEXT: v_readlane_b32 s36, v61, 20
-; SI-NEXT: v_readlane_b32 s38, v61, 18
-; SI-NEXT: v_readlane_b32 s48, v61, 16
-; SI-NEXT: v_readlane_b32 s50, v61, 14
-; SI-NEXT: v_readlane_b32 s52, v61, 12
-; SI-NEXT: v_readlane_b32 s54, v61, 10
-; SI-NEXT: s_mov_b32 s64, s76
-; SI-NEXT: s_mov_b32 s76, s6
-; SI-NEXT: s_mov_b32 s66, s58
-; SI-NEXT: s_mov_b32 s68, s60
-; SI-NEXT: s_mov_b32 s70, s78
-; SI-NEXT: s_mov_b32 s78, s10
-; SI-NEXT: s_mov_b32 s80, s88
-; SI-NEXT: s_mov_b32 s88, s14
-; SI-NEXT: s_mov_b32 s82, s90
-; SI-NEXT: s_mov_b32 s90, s16
-; SI-NEXT: s_mov_b32 s84, s92
-; SI-NEXT: s_mov_b32 s92, s4
-; SI-NEXT: s_mov_b32 s86, s94
-; SI-NEXT: s_mov_b32 s94, s74
+; SI-NEXT: v_readlane_b32 s94, v61, 32
+; SI-NEXT: v_readlane_b32 s30, v61, 18
+; SI-NEXT: v_readlane_b32 s34, v61, 16
+; SI-NEXT: v_readlane_b32 s36, v61, 14
+; SI-NEXT: v_readlane_b32 s38, v61, 12
+; SI-NEXT: v_readlane_b32 s48, v61, 10
+; SI-NEXT: v_readlane_b32 s50, v61, 8
+; SI-NEXT: v_readlane_b32 s52, v61, 6
+; SI-NEXT: v_readlane_b32 s54, v61, 4
+; SI-NEXT: v_readlane_b32 s64, v61, 2
+; SI-NEXT: v_readlane_b32 s66, v61, 0
+; SI-NEXT: s_mov_b32 s68, s46
+; SI-NEXT: s_mov_b32 s70, s42
+; SI-NEXT: s_mov_b32 s80, s78
+; SI-NEXT: s_mov_b32 s78, s8
+; SI-NEXT: s_mov_b32 s82, s88
+; SI-NEXT: s_mov_b32 s88, s10
+; SI-NEXT: s_mov_b32 s84, s90
+; SI-NEXT: s_mov_b32 s90, s12
+; SI-NEXT: s_mov_b32 s86, s92
+; SI-NEXT: s_mov_b32 s92, s16
; SI-NEXT: s_mov_b32 s96, vcc_lo
-; SI-NEXT: s_mov_b32 vcc_lo, s46
-; SI-NEXT: s_mov_b32 s98, s42
-; SI-NEXT: v_readlane_b32 s4, v61, 56
-; SI-NEXT: v_readlane_b32 s6, v61, 54
-; SI-NEXT: v_readlane_b32 s10, v61, 52
-; SI-NEXT: v_readlane_b32 s14, v61, 50
-; SI-NEXT: v_readlane_b32 s16, v61, 48
-; SI-NEXT: v_readlane_b32 s18, v61, 46
-; SI-NEXT: v_readlane_b32 s20, v61, 44
-; SI-NEXT: v_readlane_b32 s22, v61, 42
-; SI-NEXT: v_readlane_b32 s24, v61, 40
-; SI-NEXT: v_readlane_b32 s26, v61, 38
-; SI-NEXT: v_readlane_b32 s28, v61, 36
-; SI-NEXT: v_readlane_b32 s42, v61, 34
-; SI-NEXT: v_readlane_b32 s46, v61, 32
-; SI-NEXT: v_readlane_b32 s58, v61, 30
-; SI-NEXT: v_readlane_b32 s60, v61, 28
-; SI-NEXT: v_readlane_b32 s74, v61, 26
-; SI-NEXT: v_mov_b32_e32 v39, s83
-; SI-NEXT: v_mov_b32_e32 v37, s81
-; SI-NEXT: v_mov_b32_e32 v36, s71
-; SI-NEXT: v_mov_b32_e32 v28, s97
-; SI-NEXT: v_mov_b32_e32 v60, s87
-; SI-NEXT: v_mov_b32_e32 v59, s85
-; SI-NEXT: v_mov_b32_e32 v58, s99
-; SI-NEXT: v_mov_b32_e32 v57, s67
-; SI-NEXT: v_mov_b32_e32 v40, s69
-; SI-NEXT: v_readlane_b32 s31, v61, 25
-; SI-NEXT: v_readlane_b32 s35, v61, 23
-; SI-NEXT: v_readlane_b32 s37, v61, 21
-; SI-NEXT: v_readlane_b32 s39, v61, 19
-; SI-NEXT: v_readlane_b32 s49, v61, 17
-; SI-NEXT: v_readlane_b32 s51, v61, 15
-; SI-NEXT: v_readlane_b32 s53, v61, 13
-; SI-NEXT: v_readlane_b32 s55, v61, 11
-; SI-NEXT: v_readlane_b32 s5, v61, 57
-; SI-NEXT: v_readlane_b32 s7, v61, 55
-; SI-NEXT: v_readlane_b32 s11, v61, 53
-; SI-NEXT: v_readlane_b32 s15, v61, 51
-; SI-NEXT: v_readlane_b32 s17, v61, 49
-; SI-NEXT: v_readlane_b32 s19, v61, 47
-; SI-NEXT: v_readlane_b32 s21, v61, 45
-; SI-NEXT: v_readlane_b32 s23, v61, 43
-; SI-NEXT: v_readlane_b32 s25, v61, 41
-; SI-NEXT: v_readlane_b32 s27, v61, 39
-; SI-NEXT: v_readlane_b32 s29, v61, 37
-; SI-NEXT: v_readlane_b32 s43, v61, 35
-; SI-NEXT: v_readlane_b32 s47, v61, 33
-; SI-NEXT: v_readlane_b32 s59, v61, 31
-; SI-NEXT: v_readlane_b32 s61, v61, 29
-; SI-NEXT: v_readlane_b32 s75, v61, 27
+; SI-NEXT: s_mov_b32 vcc_lo, s4
+; SI-NEXT: s_mov_b32 s98, s76
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: v_readlane_b32 s4, v62, 2
+; SI-NEXT: v_readlane_b32 s6, v62, 0
+; SI-NEXT: v_readlane_b32 s8, v61, 62
+; SI-NEXT: v_readlane_b32 s10, v61, 60
+; SI-NEXT: v_readlane_b32 s12, v61, 58
+; SI-NEXT: v_readlane_b32 s16, v61, 56
+; SI-NEXT: v_readlane_b32 s18, v61, 54
+; SI-NEXT: v_readlane_b32 s20, v61, 52
+; SI-NEXT: v_readlane_b32 s22, v61, 50
+; SI-NEXT: v_readlane_b32 s24, v61, 48
+; SI-NEXT: v_readlane_b32 s26, v61, 46
+; SI-NEXT: v_readlane_b32 s28, v61, 44
+; SI-NEXT: v_readlane_b32 s42, v61, 42
+; SI-NEXT: v_readlane_b32 s46, v61, 40
+; SI-NEXT: v_readlane_b32 s56, v61, 38
+; SI-NEXT: v_readlane_b32 s62, v61, 36
+; SI-NEXT: v_readlane_b32 s74, v61, 34
+; SI-NEXT: v_mov_b32_e32 v51, s71
+; SI-NEXT: v_mov_b32_e32 v39, s97
+; SI-NEXT: v_mov_b32_e32 v38, s87
+; SI-NEXT: v_mov_b32_e32 v37, s85
+; SI-NEXT: v_mov_b32_e32 v59, s99
+; SI-NEXT: v_readlane_b32 s95, v61, 33
+; SI-NEXT: v_readlane_b32 s31, v61, 19
+; SI-NEXT: v_readlane_b32 s35, v61, 17
+; SI-NEXT: v_readlane_b32 s37, v61, 15
+; SI-NEXT: v_readlane_b32 s39, v61, 13
+; SI-NEXT: v_readlane_b32 s49, v61, 11
+; SI-NEXT: v_readlane_b32 s51, v61, 9
+; SI-NEXT: v_readlane_b32 s53, v61, 7
+; SI-NEXT: v_readlane_b32 s55, v61, 5
+; SI-NEXT: v_readlane_b32 s65, v61, 3
+; SI-NEXT: v_readlane_b32 s67, v61, 1
+; SI-NEXT: v_readlane_b32 s5, v62, 3
+; SI-NEXT: v_readlane_b32 s7, v62, 1
+; SI-NEXT: v_readlane_b32 s9, v61, 63
+; SI-NEXT: v_readlane_b32 s11, v61, 61
+; SI-NEXT: v_readlane_b32 s13, v61, 59
+; SI-NEXT: v_readlane_b32 s17, v61, 57
+; SI-NEXT: v_readlane_b32 s19, v61, 55
+; SI-NEXT: v_readlane_b32 s21, v61, 53
+; SI-NEXT: v_readlane_b32 s23, v61, 51
+; SI-NEXT: v_readlane_b32 s25, v61, 49
+; SI-NEXT: v_readlane_b32 s27, v61, 47
+; SI-NEXT: v_readlane_b32 s29, v61, 45
+; SI-NEXT: v_readlane_b32 s43, v61, 43
+; SI-NEXT: v_readlane_b32 s47, v61, 41
+; SI-NEXT: v_readlane_b32 s57, v61, 39
+; SI-NEXT: v_readlane_b32 s63, v61, 37
+; SI-NEXT: v_readlane_b32 s75, v61, 35
; SI-NEXT: .LBB37_6: ; %end
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_and_b32_e32 v38, 0xff, v38
+; SI-NEXT: v_and_b32_e32 v36, 0xff, v36
; SI-NEXT: s_lshl_b32 s5, s98, 8
-; SI-NEXT: v_or_b32_e32 v38, s5, v38
+; SI-NEXT: v_and_b32_e32 v34, 0xff, v34
+; SI-NEXT: v_lshlrev_b32_e32 v11, 8, v11
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_or_b32_e32 v36, s5, v36
; SI-NEXT: s_and_b32 s5, s96, 0xff
+; SI-NEXT: v_or_b32_e32 v11, v34, v11
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_lshlrev_b32_e32 v34, 24, v55
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: s_lshl_b32 s7, s86, 24
-; SI-NEXT: v_and_b32_e32 v38, 0xffff, v38
+; SI-NEXT: v_or_b32_e32 v9, v34, v9
+; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_and_b32_e32 v36, 0xffff, v36
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_and_b32_e32 v34, 0xff, v34
-; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v10
-; SI-NEXT: v_or_b32_e32 v38, s5, v38
-; SI-NEXT: v_or_b32_e32 v10, v34, v10
-; SI-NEXT: v_and_b32_e32 v34, 0xff, v55
-; SI-NEXT: buffer_store_dword v38, v0, s[0:3], 0 offen
-; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v54
-; SI-NEXT: v_or_b32_e32 v34, v38, v34
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_or_b32_e32 v10, v10, v34
-; SI-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:4
+; SI-NEXT: v_or_b32_e32 v9, v11, v9
+; SI-NEXT: v_or_b32_e32 v36, s5, v36
+; SI-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:4
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v10, 0xff, v33
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v33
; SI-NEXT: s_lshl_b32 s5, s84, 8
-; SI-NEXT: v_or_b32_e32 v10, s5, v10
+; SI-NEXT: v_or_b32_e32 v9, s5, v9
; SI-NEXT: s_and_b32 s5, s82, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: s_lshl_b32 s7, s80, 24
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_or_b32_e32 v10, s5, v10
-; SI-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:8
+; SI-NEXT: v_or_b32_e32 v9, s5, v9
+; SI-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:8
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v10, 0xff, v31
-; SI-NEXT: v_lshlrev_b32_e32 v31, 8, v53
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v10, v10, v31
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
-; SI-NEXT: v_or_b32_e32 v6, v6, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v10
-; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:12
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v29
+; SI-NEXT: v_lshlrev_b32_e32 v11, 8, v54
+; SI-NEXT: v_or_b32_e32 v9, v9, v11
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v53
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
+; SI-NEXT: v_or_b32_e32 v7, v7, v11
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
+; SI-NEXT: v_or_b32_e32 v7, v9, v7
+; SI-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:12
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v29
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v28
; SI-NEXT: s_lshl_b32 s5, s70, 8
-; SI-NEXT: v_or_b32_e32 v6, s5, v6
+; SI-NEXT: v_or_b32_e32 v7, s5, v7
; SI-NEXT: s_and_b32 s5, s68, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: s_lshl_b32 s7, s66, 24
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_or_b32_e32 v6, s5, v6
-; SI-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:16
+; SI-NEXT: v_or_b32_e32 v7, s5, v7
+; SI-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v27
-; SI-NEXT: v_lshlrev_b32_e32 v8, 8, v52
-; SI-NEXT: v_or_b32_e32 v6, v6, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v51
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_lshlrev_b32_e32 v4, 24, v4
-; SI-NEXT: v_or_b32_e32 v4, v4, v8
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:20
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v27
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v52
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
+; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v3
+; SI-NEXT: v_or_b32_e32 v3, v3, v7
+; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
+; SI-NEXT: v_or_b32_e32 v3, v5, v3
+; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:20
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v26
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v26
; SI-NEXT: s_lshl_b32 s5, s64, 8
-; SI-NEXT: v_or_b32_e32 v4, s5, v4
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xff
+; SI-NEXT: v_or_b32_e32 v3, s5, v3
+; SI-NEXT: s_and_b32 s5, s54, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s7, s94, 24
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
+; SI-NEXT: s_lshl_b32 s7, s52, 24
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_or_b32_e32 v4, s5, v4
-; SI-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:24
+; SI-NEXT: v_or_b32_e32 v3, s5, v3
+; SI-NEXT: buffer_store_dword v36, v0, s[0:3], 0 offen
+; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:24
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v6, 8, v50
-; SI-NEXT: v_or_b32_e32 v4, v4, v6
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: s_lshl_b32 s5, s92, 8
-; SI-NEXT: s_lshl_b32 s7, s88, 24
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v25
+; SI-NEXT: s_lshl_b32 s5, s50, 8
+; SI-NEXT: s_lshl_b32 s7, s38, 24
; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
; SI-NEXT: s_lshl_b32 s4, s4, 24
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
@@ -44369,140 +44370,147 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_readlane_b32 s65, v63, 15
; SI-NEXT: v_readlane_b32 s64, v63, 14
; SI-NEXT: v_readlane_b32 s55, v63, 13
+; SI-NEXT: v_readlane_b32 s54, v63, 12
; SI-NEXT: v_readlane_b32 s53, v63, 11
+; SI-NEXT: v_readlane_b32 s52, v63, 10
; SI-NEXT: v_readlane_b32 s51, v63, 9
+; SI-NEXT: v_readlane_b32 s50, v63, 8
; SI-NEXT: v_readlane_b32 s49, v63, 7
; SI-NEXT: v_readlane_b32 s39, v63, 5
+; SI-NEXT: v_readlane_b32 s38, v63, 4
; SI-NEXT: v_readlane_b32 s37, v63, 3
; SI-NEXT: v_readlane_b32 s35, v63, 1
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
-; SI-NEXT: v_or_b32_e32 v3, v6, v3
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:28
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v24
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s90, 0xff
+; SI-NEXT: s_and_b32 s5, s48, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:32
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v23
-; SI-NEXT: s_lshl_b32 s5, s78, 8
-; SI-NEXT: s_lshl_b32 s7, s54, 24
-; SI-NEXT: v_readlane_b32 s54, v63, 12
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: s_lshl_b32 s5, s36, 8
+; SI-NEXT: s_lshl_b32 s7, s30, 24
+; SI-NEXT: v_readlane_b32 s30, v63, 34
+; SI-NEXT: v_readlane_b32 s31, v63, 35
+; SI-NEXT: v_readlane_b32 s48, v63, 6
+; SI-NEXT: v_readlane_b32 s36, v63, 2
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:36
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v22
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s76, 0xff
+; SI-NEXT: s_and_b32 s5, s34, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:40
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v21
-; SI-NEXT: s_lshl_b32 s5, s52, 8
-; SI-NEXT: s_lshl_b32 s7, s48, 24
-; SI-NEXT: v_readlane_b32 s52, v63, 10
-; SI-NEXT: v_readlane_b32 s48, v63, 6
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 8
+; SI-NEXT: s_lshl_b32 s7, s90, 24
+; SI-NEXT: v_readlane_b32 s34, v63, 0
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:44
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v20
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s50, 0xff
+; SI-NEXT: s_and_b32 s5, s92, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:48
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v19
-; SI-NEXT: s_lshl_b32 s5, s38, 8
-; SI-NEXT: s_lshl_b32 s7, s34, 24
-; SI-NEXT: v_readlane_b32 s50, v63, 8
-; SI-NEXT: v_readlane_b32 s38, v63, 4
-; SI-NEXT: v_readlane_b32 s34, v63, 0
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v18
+; SI-NEXT: s_lshl_b32 s5, s88, 8
+; SI-NEXT: s_lshl_b32 s7, s76, 24
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:52
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v18
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v19
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s36, 0xff
+; SI-NEXT: s_and_b32 s5, s78, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:56
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v17
-; SI-NEXT: s_lshl_b32 s5, s30, 8
-; SI-NEXT: s_lshl_b32 s7, s60, 24
-; SI-NEXT: v_readlane_b32 s30, v63, 34
-; SI-NEXT: v_readlane_b32 s31, v63, 35
-; SI-NEXT: v_readlane_b32 s36, v63, 2
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v51
+; SI-NEXT: s_lshl_b32 s5, s94, 8
+; SI-NEXT: s_lshl_b32 s7, s62, 24
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:60
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v16
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v17
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: s_and_b32 s5, s74, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
@@ -44510,69 +44518,67 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:64
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v48
-; SI-NEXT: s_lshl_b32 s5, s58, 8
-; SI-NEXT: s_lshl_b32 s7, s42, 24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v49
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v14
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v50
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v49
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v48
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:68
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v14
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v15
+; SI-NEXT: s_lshl_b32 s5, s56, 8
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: s_and_b32 s5, s46, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s7, s42, 24
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:72
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v39
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v37
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v36
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v39
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v38
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v37
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:76
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v12
-; SI-NEXT: s_lshl_b32 s5, s62, 8
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v13
+; SI-NEXT: s_lshl_b32 s5, s44, 8
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s56, 0xff
+; SI-NEXT: s_and_b32 s5, s72, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s7, s44, 24
+; SI-NEXT: s_lshl_b32 s7, s60, 24
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:80
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v35
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v32
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v30
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v35
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v32
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v31
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:84
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: s_lshl_b32 s5, s72, 8
+; SI-NEXT: s_lshl_b32 s5, s58, 8
; SI-NEXT: s_lshl_b32 s7, s28, 24
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v28
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v59
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v30
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v59
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_or_b32_e32 v3, s5, v3
@@ -44583,20 +44589,19 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:88
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v60
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v8
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v60
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:92
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: s_lshl_b32 s5, s26, 8
; SI-NEXT: s_lshl_b32 s7, s22, 24
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v58
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v56
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v58
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_or_b32_e32 v3, s5, v3
@@ -44607,19 +44612,20 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:96
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v7
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v6
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v57
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v56
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v6, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:100
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; SI-NEXT: s_lshl_b32 s5, s20, 8
; SI-NEXT: s_lshl_b32 s7, s16, 24
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v45
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_or_b32_e32 v3, s5, v3
@@ -44630,11 +44636,11 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:104
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v5
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v4
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v47
; SI-NEXT: v_or_b32_e32 v3, v3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v46
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v45
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: v_or_b32_e32 v4, v5, v4
; SI-NEXT: v_or_b32_e32 v3, v3, v4
@@ -68903,183 +68909,213 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -69088,9 +69124,9 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -69102,7 +69138,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -69110,11 +69146,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -69125,7 +69161,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -69134,11 +69170,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -69149,7 +69185,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -69158,11 +69194,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -69173,7 +69209,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -69182,11 +69218,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -69194,47 +69230,56 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -69242,10 +69287,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -69253,12 +69299,14 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -69266,10 +69314,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -69277,12 +69326,14 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -69290,10 +69341,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -69301,12 +69353,14 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -69314,10 +69368,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -69333,7 +69388,8 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -69381,24 +69437,8 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -72100,195 +72140,215 @@ define inreg <32 x float> @bitcast_v64i16_to_v32f32_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB51_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB51_3
; SI-NEXT: .LBB51_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB51_3: ; %Flow
@@ -72299,168 +72359,188 @@ define inreg <32 x float> @bitcast_v64i16_to_v32f32_scalar(<64 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB51_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -72493,23 +72573,7 @@ define inreg <32 x float> @bitcast_v64i16_to_v32f32_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -93219,139 +93283,139 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: v_writelane_b32 v33, s55, 13
; SI-NEXT: v_writelane_b32 v33, s64, 14
; SI-NEXT: v_writelane_b32 v33, s65, 15
-; SI-NEXT: v_writelane_b32 v33, s66, 16
-; SI-NEXT: v_writelane_b32 v33, s67, 17
-; SI-NEXT: v_writelane_b32 v33, s68, 18
-; SI-NEXT: v_writelane_b32 v33, s69, 19
-; SI-NEXT: v_writelane_b32 v33, s70, 20
-; SI-NEXT: v_writelane_b32 v33, s71, 21
-; SI-NEXT: v_writelane_b32 v33, s80, 22
-; SI-NEXT: v_writelane_b32 v33, s81, 23
-; SI-NEXT: v_writelane_b32 v33, s82, 24
-; SI-NEXT: v_writelane_b32 v33, s83, 25
-; SI-NEXT: v_writelane_b32 v33, s84, 26
-; SI-NEXT: v_writelane_b32 v33, s85, 27
-; SI-NEXT: v_writelane_b32 v33, s86, 28
-; SI-NEXT: v_writelane_b32 v33, s87, 29
-; SI-NEXT: v_writelane_b32 v33, s96, 30
-; SI-NEXT: v_writelane_b32 v33, s97, 31
-; SI-NEXT: v_writelane_b32 v33, s98, 32
-; SI-NEXT: v_writelane_b32 v33, s30, 33
-; SI-NEXT: v_writelane_b32 v33, s31, 34
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_readfirstlane_b32 s4, v17
-; SI-NEXT: v_readfirstlane_b32 s30, v16
-; SI-NEXT: v_readfirstlane_b32 s31, v15
-; SI-NEXT: v_readfirstlane_b32 s34, v14
-; SI-NEXT: v_readfirstlane_b32 s35, v13
-; SI-NEXT: v_readfirstlane_b32 s36, v12
-; SI-NEXT: v_readfirstlane_b32 s37, v11
-; SI-NEXT: v_readfirstlane_b32 s38, v10
-; SI-NEXT: v_readfirstlane_b32 s39, v9
-; SI-NEXT: v_readfirstlane_b32 s48, v8
-; SI-NEXT: v_readfirstlane_b32 s49, v7
-; SI-NEXT: v_readfirstlane_b32 s50, v6
-; SI-NEXT: v_readfirstlane_b32 s51, v5
-; SI-NEXT: v_readfirstlane_b32 s52, v4
-; SI-NEXT: v_readfirstlane_b32 s53, v3
-; SI-NEXT: v_readfirstlane_b32 s54, v2
-; SI-NEXT: v_readfirstlane_b32 s55, v1
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_readfirstlane_b32 s64, v0
+; SI-NEXT: v_writelane_b32 v33, s30, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 17
+; SI-NEXT: v_readfirstlane_b32 s7, v17
; SI-NEXT: ; implicit-def: $vgpr34 : SGPR spill to VGPR lane
+; SI-NEXT: v_readfirstlane_b32 s9, v16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v34, s4, 0
+; SI-NEXT: v_writelane_b32 v34, s7, 0
+; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: v_writelane_b32 v34, s9, 1
+; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v34, s11, 2
+; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v34, s13, 3
+; SI-NEXT: v_readfirstlane_b32 s41, v12
+; SI-NEXT: v_writelane_b32 v34, s15, 4
+; SI-NEXT: v_readfirstlane_b32 s43, v11
+; SI-NEXT: v_writelane_b32 v34, s41, 5
+; SI-NEXT: v_readfirstlane_b32 s45, v10
+; SI-NEXT: v_writelane_b32 v34, s43, 6
+; SI-NEXT: v_readfirstlane_b32 s47, v9
+; SI-NEXT: v_writelane_b32 v34, s45, 7
+; SI-NEXT: v_readfirstlane_b32 s57, v8
+; SI-NEXT: v_writelane_b32 v34, s47, 8
+; SI-NEXT: v_readfirstlane_b32 s59, v7
+; SI-NEXT: v_writelane_b32 v34, s57, 9
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_writelane_b32 v34, s59, 10
+; SI-NEXT: v_readfirstlane_b32 s63, v5
+; SI-NEXT: v_writelane_b32 v34, s61, 11
+; SI-NEXT: v_readfirstlane_b32 s73, v4
+; SI-NEXT: v_writelane_b32 v34, s63, 12
+; SI-NEXT: v_readfirstlane_b32 s6, v3
+; SI-NEXT: v_writelane_b32 v34, s73, 13
+; SI-NEXT: v_readfirstlane_b32 s5, v2
+; SI-NEXT: v_writelane_b32 v34, s6, 14
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: v_readfirstlane_b32 s8, v18
+; SI-NEXT: v_writelane_b32 v34, s5, 15
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_readfirstlane_b32 s8, v0
+; SI-NEXT: v_writelane_b32 v34, s4, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 17
; SI-NEXT: s_cbranch_scc0 .LBB61_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s8, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s4, 16
-; SI-NEXT: s_and_b32 s4, s50, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 2
-; SI-NEXT: s_lshl_b32 s4, s50, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 1
-; SI-NEXT: s_and_b32 s4, s51, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 4
-; SI-NEXT: s_lshl_b32 s4, s51, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 3
-; SI-NEXT: s_and_b32 s4, s52, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 6
-; SI-NEXT: s_lshl_b32 s4, s52, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 5
-; SI-NEXT: s_and_b32 s4, s53, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 8
-; SI-NEXT: s_lshl_b32 s4, s53, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 7
-; SI-NEXT: s_and_b32 s4, s55, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 10
-; SI-NEXT: s_lshl_b32 s4, s55, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 9
-; SI-NEXT: s_and_b32 s4, s64, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 12
-; SI-NEXT: s_lshl_b32 s4, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 11
-; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 14
-; SI-NEXT: s_lshl_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 13
-; SI-NEXT: s_and_b32 s4, s28, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 16
-; SI-NEXT: s_lshl_b32 s4, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 15
+; SI-NEXT: s_and_b32 s88, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s4, 16
; SI-NEXT: s_and_b32 s4, s27, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 18
+; SI-NEXT: v_writelane_b32 v34, s4, 19
; SI-NEXT: s_lshl_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 17
+; SI-NEXT: v_writelane_b32 v34, s4, 18
; SI-NEXT: s_and_b32 s4, s26, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: v_writelane_b32 v34, s4, 21
; SI-NEXT: s_lshl_b32 s4, s26, 16
-; SI-NEXT: s_and_b32 s10, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s30, 16
-; SI-NEXT: s_and_b32 s12, s31, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s31, 16
-; SI-NEXT: s_and_b32 s14, s34, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s34, 16
-; SI-NEXT: s_and_b32 s40, s35, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s35, 16
-; SI-NEXT: s_and_b32 s42, s36, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s36, 16
-; SI-NEXT: s_and_b32 s44, s37, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s37, 16
-; SI-NEXT: s_and_b32 s46, s38, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s38, 16
-; SI-NEXT: s_and_b32 s56, s39, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s39, 16
-; SI-NEXT: s_and_b32 s58, s48, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s48, 16
-; SI-NEXT: s_and_b32 s60, s49, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s49, 16
-; SI-NEXT: s_and_b32 s78, s54, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s54, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 19
-; SI-NEXT: s_and_b32 s62, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s25, 16
-; SI-NEXT: s_and_b32 s72, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s24, 16
-; SI-NEXT: s_and_b32 s74, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s23, 16
-; SI-NEXT: s_and_b32 s76, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s75, s22, 16
-; SI-NEXT: s_and_b32 s88, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s79, s21, 16
-; SI-NEXT: s_and_b32 s90, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s89, s20, 16
-; SI-NEXT: s_and_b32 s92, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s91, s19, 16
-; SI-NEXT: s_and_b32 s94, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s93, s18, 16
-; SI-NEXT: s_and_b32 vcc_lo, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s95, s17, 16
-; SI-NEXT: s_and_b32 s6, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_hi, s16, 16
+; SI-NEXT: s_and_b32 s55, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_and_b32 s10, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_and_b32 s12, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_and_b32 s14, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_and_b32 s40, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_and_b32 s42, s41, 0xffff0000
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_and_b32 s44, s43, 0xffff0000
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s46, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_and_b32 s56, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_and_b32 s58, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_and_b32 s60, s59, 0xffff0000
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_and_b32 s62, s61, 0xffff0000
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_and_b32 s72, s63, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_and_b32 s74, s73, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s73, 16
+; SI-NEXT: s_and_b32 s76, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s75, s6, 16
+; SI-NEXT: s_and_b32 s78, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s5, 16
+; SI-NEXT: s_and_b32 s90, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s8, 16
+; SI-NEXT: s_and_b32 s92, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s29, 16
+; SI-NEXT: s_and_b32 s94, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s28, 16
+; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s6, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s8, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s16, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB61_3
; SI-NEXT: .LBB61_2:
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr50
+; SI-NEXT: ; implicit-def: $sgpr49
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr95
@@ -93364,6 +93428,8 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr88
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr75
; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $sgpr73
@@ -93372,8 +93438,6 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr61
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr59
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $sgpr57
@@ -93395,39 +93459,7 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
; SI-NEXT: ; implicit-def: $sgpr4
@@ -93439,68 +93471,43 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s65, s63
-; SI-NEXT: s_mov_b32 s66, s72
-; SI-NEXT: s_mov_b32 s67, s73
-; SI-NEXT: s_mov_b32 s68, s74
-; SI-NEXT: s_mov_b32 s69, s75
-; SI-NEXT: s_mov_b32 s70, s76
-; SI-NEXT: s_mov_b32 s71, s79
-; SI-NEXT: s_mov_b32 s80, s88
-; SI-NEXT: s_mov_b32 s81, s89
-; SI-NEXT: s_mov_b32 s82, s90
-; SI-NEXT: s_mov_b32 s83, s91
-; SI-NEXT: s_mov_b32 s84, s92
-; SI-NEXT: s_mov_b32 s85, s93
-; SI-NEXT: s_mov_b32 s86, s94
-; SI-NEXT: s_mov_b32 s87, s95
-; SI-NEXT: s_mov_b32 s96, vcc_lo
-; SI-NEXT: s_mov_b32 vcc_lo, s61
-; SI-NEXT: s_mov_b32 s97, vcc_hi
-; SI-NEXT: s_mov_b32 vcc_hi, s62
-; SI-NEXT: s_mov_b32 s98, s6
+; SI-NEXT: s_mov_b32 s64, s30
+; SI-NEXT: s_mov_b32 s30, s6
+; SI-NEXT: s_mov_b32 s65, s8
; SI-NEXT: s_mov_b32 s4, s7
-; SI-NEXT: s_mov_b32 s5, s8
-; SI-NEXT: s_mov_b32 s7, s9
-; SI-NEXT: s_mov_b32 s8, s10
-; SI-NEXT: s_mov_b32 s9, s11
-; SI-NEXT: s_mov_b32 s10, s12
-; SI-NEXT: s_mov_b32 s11, s13
-; SI-NEXT: s_mov_b32 s12, s14
-; SI-NEXT: s_mov_b32 s13, s15
-; SI-NEXT: s_mov_b32 s14, s40
-; SI-NEXT: s_mov_b32 s15, s41
-; SI-NEXT: s_mov_b32 s40, s42
-; SI-NEXT: s_mov_b32 s41, s43
-; SI-NEXT: s_mov_b32 s42, s44
-; SI-NEXT: s_mov_b32 s43, s45
-; SI-NEXT: s_mov_b32 s44, s46
-; SI-NEXT: s_mov_b32 s45, s47
-; SI-NEXT: s_mov_b32 s46, s56
-; SI-NEXT: s_mov_b32 s47, s57
-; SI-NEXT: s_mov_b32 s56, s58
-; SI-NEXT: s_mov_b32 s57, s59
-; SI-NEXT: s_mov_b32 s58, s60
-; SI-NEXT: v_readlane_b32 s59, v34, 1
-; SI-NEXT: v_readlane_b32 s60, v34, 2
-; SI-NEXT: v_readlane_b32 s61, v34, 3
-; SI-NEXT: v_readlane_b32 s62, v34, 4
-; SI-NEXT: v_readlane_b32 s63, v34, 5
-; SI-NEXT: v_readlane_b32 s72, v34, 6
-; SI-NEXT: v_readlane_b32 s73, v34, 7
-; SI-NEXT: v_readlane_b32 s74, v34, 8
-; SI-NEXT: v_readlane_b32 s6, v34, 9
-; SI-NEXT: v_readlane_b32 s75, v34, 10
-; SI-NEXT: v_readlane_b32 s76, v34, 11
-; SI-NEXT: v_readlane_b32 s79, v34, 12
-; SI-NEXT: v_readlane_b32 s88, v34, 13
-; SI-NEXT: v_readlane_b32 s89, v34, 14
-; SI-NEXT: v_readlane_b32 s90, v34, 15
-; SI-NEXT: v_readlane_b32 s91, v34, 16
-; SI-NEXT: v_readlane_b32 s92, v34, 17
-; SI-NEXT: v_readlane_b32 s93, v34, 18
-; SI-NEXT: v_readlane_b32 s94, v34, 19
-; SI-NEXT: v_readlane_b32 s95, v34, 20
+; SI-NEXT: s_mov_b32 s5, s55
+; SI-NEXT: s_mov_b32 s55, s9
+; SI-NEXT: s_mov_b32 s9, s10
+; SI-NEXT: s_mov_b32 s10, s11
+; SI-NEXT: s_mov_b32 s11, s12
+; SI-NEXT: s_mov_b32 s12, s13
+; SI-NEXT: s_mov_b32 s13, s14
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s40, s41
+; SI-NEXT: s_mov_b32 s41, s42
+; SI-NEXT: s_mov_b32 s42, s43
+; SI-NEXT: s_mov_b32 s43, s44
+; SI-NEXT: s_mov_b32 s44, s45
+; SI-NEXT: s_mov_b32 s45, s46
+; SI-NEXT: s_mov_b32 s46, s47
+; SI-NEXT: s_mov_b32 s47, s56
+; SI-NEXT: s_mov_b32 s56, s57
+; SI-NEXT: s_mov_b32 s57, s58
+; SI-NEXT: s_mov_b32 s58, s59
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_mov_b32 s60, s61
+; SI-NEXT: s_mov_b32 s61, s62
+; SI-NEXT: s_mov_b32 s62, s63
+; SI-NEXT: s_mov_b32 s63, s72
+; SI-NEXT: s_mov_b32 s72, s73
+; SI-NEXT: s_mov_b32 s73, s74
+; SI-NEXT: s_mov_b32 s74, s75
+; SI-NEXT: s_mov_b32 s75, s76
+; SI-NEXT: v_readlane_b32 s6, v34, 18
+; SI-NEXT: v_readlane_b32 s7, v34, 19
+; SI-NEXT: v_readlane_b32 s8, v34, 20
+; SI-NEXT: v_readlane_b32 s76, v34, 21
; SI-NEXT: s_cbranch_scc1 .LBB61_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_u32 s16, s16, 3
@@ -93513,241 +93520,241 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: s_addc_u32 s23, s23, 0
; SI-NEXT: s_add_u32 s24, s24, 3
; SI-NEXT: s_addc_u32 s25, s25, 0
-; SI-NEXT: s_add_u32 s26, s26, 3
-; SI-NEXT: s_addc_u32 s27, s27, 0
-; SI-NEXT: s_add_u32 s28, s28, 3
-; SI-NEXT: s_addc_u32 s29, s29, 0
-; SI-NEXT: s_add_u32 s76, s64, 3
-; SI-NEXT: s_addc_u32 s6, s55, 0
-; SI-NEXT: s_add_u32 s75, s54, 3
-; SI-NEXT: s_addc_u32 s73, s53, 0
-; SI-NEXT: s_add_u32 s63, s52, 3
-; SI-NEXT: s_addc_u32 s61, s51, 0
-; SI-NEXT: s_add_u32 s59, s50, 3
-; SI-NEXT: s_addc_u32 s57, s49, 0
-; SI-NEXT: s_add_u32 s47, s48, 3
-; SI-NEXT: s_addc_u32 s45, s39, 0
-; SI-NEXT: s_add_u32 s43, s38, 3
-; SI-NEXT: s_addc_u32 s41, s37, 0
-; SI-NEXT: s_add_u32 s15, s36, 3
-; SI-NEXT: s_addc_u32 s13, s35, 0
-; SI-NEXT: s_add_u32 s11, s34, 3
-; SI-NEXT: s_addc_u32 s9, s31, 0
-; SI-NEXT: s_add_u32 s7, s30, 3
+; SI-NEXT: s_add_u32 s8, s26, 3
+; SI-NEXT: s_addc_u32 s6, s27, 0
+; SI-NEXT: s_add_u32 s7, s28, 3
+; SI-NEXT: s_addc_u32 s26, s29, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 17
+; SI-NEXT: s_add_u32 s27, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 16
+; SI-NEXT: s_addc_u32 s28, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 15
+; SI-NEXT: s_add_u32 s29, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 14
+; SI-NEXT: s_addc_u32 s74, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 13
+; SI-NEXT: s_add_u32 s72, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 12
+; SI-NEXT: s_addc_u32 s62, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 11
+; SI-NEXT: s_add_u32 s60, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 10
+; SI-NEXT: s_addc_u32 s58, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 9
+; SI-NEXT: s_add_u32 s56, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 8
+; SI-NEXT: s_addc_u32 s46, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 7
+; SI-NEXT: s_add_u32 s44, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 6
+; SI-NEXT: s_addc_u32 s42, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 5
+; SI-NEXT: s_add_u32 s40, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 4
+; SI-NEXT: s_addc_u32 s14, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 3
+; SI-NEXT: s_add_u32 s12, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 2
+; SI-NEXT: s_addc_u32 s10, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 1
+; SI-NEXT: s_add_u32 s11, s4, 3
; SI-NEXT: v_readlane_b32 s4, v34, 0
; SI-NEXT: s_addc_u32 s4, s4, 0
; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_and_b32 s8, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s10, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s12, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_and_b32 s14, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s13, 16
-; SI-NEXT: s_and_b32 s40, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s15, 16
-; SI-NEXT: s_and_b32 s42, s41, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s41, 16
-; SI-NEXT: s_and_b32 s44, s43, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_and_b32 s46, s45, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s45, 16
-; SI-NEXT: s_and_b32 s56, s47, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s47, 16
-; SI-NEXT: s_and_b32 s58, s57, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s57, 16
-; SI-NEXT: s_and_b32 s60, s59, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s59, 16
-; SI-NEXT: s_and_b32 s62, s61, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s61, 16
-; SI-NEXT: s_and_b32 s72, s63, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s63, 16
-; SI-NEXT: s_and_b32 s74, s73, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s73, 16
-; SI-NEXT: s_and_b32 s78, s75, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s75, 16
-; SI-NEXT: s_and_b32 s75, s6, 0xffff0000
+; SI-NEXT: s_and_b32 s9, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s55, s11, 16
+; SI-NEXT: s_and_b32 s11, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s13, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_and_b32 s15, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_and_b32 s41, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s43, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s45, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_and_b32 s47, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_and_b32 s57, s56, 0xffff0000
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_and_b32 s59, s58, 0xffff0000
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_and_b32 s61, s60, 0xffff0000
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_and_b32 s63, s62, 0xffff0000
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_and_b32 s73, s72, 0xffff0000
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_and_b32 s75, s74, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s74, 16
+; SI-NEXT: s_and_b32 s78, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s29, 16
+; SI-NEXT: s_and_b32 s88, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s28, 16
+; SI-NEXT: s_and_b32 s90, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s27, 16
+; SI-NEXT: s_and_b32 s92, s26, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s26, 16
+; SI-NEXT: s_and_b32 s94, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s7, 16
+; SI-NEXT: s_and_b32 s7, s6, 0xffff0000
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_and_b32 s79, s76, 0xffff0000
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_and_b32 s89, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s88, s29, 16
-; SI-NEXT: s_and_b32 s91, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s28, 16
-; SI-NEXT: s_and_b32 s93, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s92, s27, 16
-; SI-NEXT: s_and_b32 s95, s26, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s26, 16
-; SI-NEXT: s_and_b32 vcc_hi, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s25, 16
-; SI-NEXT: s_and_b32 s66, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s65, s24, 16
-; SI-NEXT: s_and_b32 s68, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s23, 16
-; SI-NEXT: s_and_b32 s70, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s22, 16
-; SI-NEXT: s_and_b32 s80, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s21, 16
-; SI-NEXT: s_and_b32 s82, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s20, 16
-; SI-NEXT: s_and_b32 s84, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s83, s19, 16
-; SI-NEXT: s_and_b32 s86, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s85, s18, 16
-; SI-NEXT: s_and_b32 s96, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s87, s17, 16
-; SI-NEXT: s_and_b32 s98, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s97, s16, 16
+; SI-NEXT: s_and_b32 s76, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s30, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s65, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s16, 16
; SI-NEXT: .LBB61_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s98
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s97
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s64
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s96
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s31
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s34
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s36
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s38
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s48
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s49
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s50
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s52
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s54
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s30
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, vcc_hi
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_hi
-; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_lo
+; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s95
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s76
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s8
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s7
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s6
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s93
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s91
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s90
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s89
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s79
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s77
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s75
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s73
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s63
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s61
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s59
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s57
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s47
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s45
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s15
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s13
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s55
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
-; SI-NEXT: v_readlane_b32 s30, v33, 33
+; SI-NEXT: v_readlane_b32 s30, v33, 16
; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
-; SI-NEXT: v_readlane_b32 s31, v33, 34
-; SI-NEXT: v_readlane_b32 s98, v33, 32
-; SI-NEXT: v_readlane_b32 s97, v33, 31
-; SI-NEXT: v_readlane_b32 s96, v33, 30
-; SI-NEXT: v_readlane_b32 s87, v33, 29
-; SI-NEXT: v_readlane_b32 s86, v33, 28
-; SI-NEXT: v_readlane_b32 s85, v33, 27
-; SI-NEXT: v_readlane_b32 s84, v33, 26
-; SI-NEXT: v_readlane_b32 s83, v33, 25
-; SI-NEXT: v_readlane_b32 s82, v33, 24
-; SI-NEXT: v_readlane_b32 s81, v33, 23
-; SI-NEXT: v_readlane_b32 s80, v33, 22
-; SI-NEXT: v_readlane_b32 s71, v33, 21
-; SI-NEXT: v_readlane_b32 s70, v33, 20
-; SI-NEXT: v_readlane_b32 s69, v33, 19
-; SI-NEXT: v_readlane_b32 s68, v33, 18
-; SI-NEXT: v_readlane_b32 s67, v33, 17
-; SI-NEXT: v_readlane_b32 s66, v33, 16
+; SI-NEXT: v_readlane_b32 s31, v33, 17
; SI-NEXT: v_readlane_b32 s65, v33, 15
; SI-NEXT: v_readlane_b32 s64, v33, 14
; SI-NEXT: v_readlane_b32 s55, v33, 13
@@ -103007,183 +103014,213 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB67_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB67_3
; SI-NEXT: .LBB67_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB67_3: ; %Flow
@@ -103192,9 +103229,9 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB67_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -103206,7 +103243,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -103214,11 +103251,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -103229,7 +103266,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -103238,11 +103275,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -103253,7 +103290,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -103262,11 +103299,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -103277,7 +103314,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -103286,11 +103323,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -103298,47 +103335,56 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -103346,10 +103392,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -103357,12 +103404,14 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -103370,10 +103419,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -103381,12 +103431,14 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -103394,10 +103446,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -103405,12 +103458,14 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -103418,10 +103473,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -103437,7 +103493,8 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -103485,24 +103542,8 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB67_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -106104,195 +106145,215 @@ define inreg <16 x i64> @bitcast_v64i16_to_v16i64_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB71_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB71_3
; SI-NEXT: .LBB71_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB71_3: ; %Flow
@@ -106303,168 +106364,188 @@ define inreg <16 x i64> @bitcast_v64i16_to_v16i64_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB71_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -106497,23 +106578,7 @@ define inreg <16 x i64> @bitcast_v64i16_to_v16i64_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -136232,183 +136297,213 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB83_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB83_3
; SI-NEXT: .LBB83_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB83_3: ; %Flow
@@ -136417,9 +136512,9 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB83_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -136431,7 +136526,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -136439,11 +136534,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -136454,7 +136549,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -136463,11 +136558,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -136478,7 +136573,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -136487,11 +136582,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -136502,7 +136597,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -136511,11 +136606,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -136523,47 +136618,56 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -136571,10 +136675,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -136582,12 +136687,14 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -136595,10 +136702,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -136606,12 +136714,14 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -136619,10 +136729,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -136630,12 +136741,14 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -136643,10 +136756,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -136662,7 +136776,8 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -136710,24 +136825,8 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB83_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -139319,195 +139418,215 @@ define inreg <16 x double> @bitcast_v64i16_to_v16f64_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB87_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB87_3
; SI-NEXT: .LBB87_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB87_3: ; %Flow
@@ -139518,168 +139637,188 @@ define inreg <16 x double> @bitcast_v64i16_to_v16f64_scalar(<64 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB87_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -139712,23 +139851,7 @@ define inreg <16 x double> @bitcast_v64i16_to_v16f64_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -146663,1522 +146786,1576 @@ define inreg <64 x bfloat> @bitcast_v128i8_to_v64bf16_scalar(<128 x i8> inreg %a
; SI-NEXT: v_writelane_b32 v45, s99, 33
; SI-NEXT: v_writelane_b32 v45, s30, 34
; SI-NEXT: v_writelane_b32 v45, s31, 35
-; SI-NEXT: ; implicit-def: $vgpr56 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s4, v30
-; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: v_writelane_b32 v56, s29, 0
-; SI-NEXT: v_writelane_b32 v56, s28, 1
-; SI-NEXT: v_writelane_b32 v56, s27, 2
-; SI-NEXT: v_writelane_b32 v56, s26, 3
-; SI-NEXT: v_writelane_b32 v56, s25, 4
-; SI-NEXT: v_writelane_b32 v56, s24, 5
-; SI-NEXT: v_writelane_b32 v56, s23, 6
-; SI-NEXT: v_writelane_b32 v56, s22, 7
-; SI-NEXT: v_writelane_b32 v56, s21, 8
-; SI-NEXT: v_writelane_b32 v56, s4, 9
-; SI-NEXT: v_readfirstlane_b32 s4, v29
-; SI-NEXT: v_writelane_b32 v56, s4, 10
-; SI-NEXT: v_readfirstlane_b32 s4, v28
-; SI-NEXT: v_writelane_b32 v56, s4, 11
-; SI-NEXT: v_readfirstlane_b32 s4, v27
-; SI-NEXT: v_writelane_b32 v56, s4, 12
-; SI-NEXT: v_readfirstlane_b32 s4, v26
-; SI-NEXT: v_writelane_b32 v56, s4, 13
-; SI-NEXT: v_readfirstlane_b32 s4, v25
-; SI-NEXT: v_writelane_b32 v56, s4, 14
-; SI-NEXT: v_readfirstlane_b32 s4, v24
-; SI-NEXT: v_writelane_b32 v56, s4, 15
-; SI-NEXT: v_readfirstlane_b32 s4, v23
+; SI-NEXT: v_readfirstlane_b32 s91, v23
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:332
-; SI-NEXT: s_mov_b32 s6, s19
-; SI-NEXT: v_writelane_b32 v56, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s4, v22
-; SI-NEXT: v_readfirstlane_b32 s19, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:328
-; SI-NEXT: v_writelane_b32 v56, s4, 17
-; SI-NEXT: v_readfirstlane_b32 s4, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:324
-; SI-NEXT: v_writelane_b32 v56, s4, 18
-; SI-NEXT: v_readfirstlane_b32 s4, v20
-; SI-NEXT: v_writelane_b32 v56, s4, 19
-; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:320
+; SI-NEXT: v_readfirstlane_b32 s31, v22
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:328
+; SI-NEXT: v_readfirstlane_b32 s94, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:324
+; SI-NEXT: v_readfirstlane_b32 s63, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:320
+; SI-NEXT: v_readfirstlane_b32 s75, v25
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:316
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:312
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:308
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:304
-; SI-NEXT: v_writelane_b32 v56, s4, 20
-; SI-NEXT: v_readfirstlane_b32 s75, v15
-; SI-NEXT: v_readfirstlane_b32 s44, v16
-; SI-NEXT: v_readfirstlane_b32 s27, v12
-; SI-NEXT: v_readfirstlane_b32 s52, v0
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:244
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
-; SI-NEXT: v_readfirstlane_b32 s89, v10
-; SI-NEXT: v_readfirstlane_b32 s62, v9
-; SI-NEXT: v_readfirstlane_b32 s92, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:300
-; SI-NEXT: v_readfirstlane_b32 s88, v14
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:296
-; SI-NEXT: v_readfirstlane_b32 s59, v13
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:292
-; SI-NEXT: v_readfirstlane_b32 s77, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:288
-; SI-NEXT: s_mov_b32 s7, s20
-; SI-NEXT: v_readfirstlane_b32 s79, v8
-; SI-NEXT: v_readfirstlane_b32 s12, v7
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:284
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:264
-; SI-NEXT: v_readfirstlane_b32 s46, v6
+; SI-NEXT: ; implicit-def: $vgpr56 : SGPR spill to VGPR lane
+; SI-NEXT: v_readfirstlane_b32 s80, v21
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: v_writelane_b32 v56, s19, 0
+; SI-NEXT: v_writelane_b32 v56, s17, 1
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:312
+; SI-NEXT: v_writelane_b32 v56, s16, 2
+; SI-NEXT: v_readfirstlane_b32 s35, v18
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:308
+; SI-NEXT: v_readfirstlane_b32 s4, v7
+; SI-NEXT: v_readfirstlane_b32 s39, v16
+; SI-NEXT: v_readfirstlane_b32 s37, v15
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:304
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:300
+; SI-NEXT: v_readfirstlane_b32 s54, v11
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:296
+; SI-NEXT: v_writelane_b32 v56, s4, 3
+; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_readfirstlane_b32 s96, v9
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292
+; SI-NEXT: v_writelane_b32 v56, s4, 4
+; SI-NEXT: v_readfirstlane_b32 s4, v5
+; SI-NEXT: v_readfirstlane_b32 s95, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:288
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:284
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:280
-; SI-NEXT: v_readfirstlane_b32 s22, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276
-; SI-NEXT: v_readfirstlane_b32 s36, v4
+; SI-NEXT: v_writelane_b32 v56, s4, 5
+; SI-NEXT: v_readfirstlane_b32 s4, v4
+; SI-NEXT: v_writelane_b32 v56, s4, 6
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272
-; SI-NEXT: v_readfirstlane_b32 s20, v3
+; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268
+; SI-NEXT: v_readfirstlane_b32 s34, v10
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:264
+; SI-NEXT: v_readfirstlane_b32 s38, v13
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:260
+; SI-NEXT: v_writelane_b32 v56, s4, 7
+; SI-NEXT: v_readfirstlane_b32 s48, v14
+; SI-NEXT: v_readfirstlane_b32 s92, v12
+; SI-NEXT: v_readfirstlane_b32 s90, v0
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:244
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
+; SI-NEXT: v_readfirstlane_b32 s79, v24
+; SI-NEXT: v_readfirstlane_b32 s62, v28
+; SI-NEXT: v_readfirstlane_b32 s77, v17
+; SI-NEXT: s_mov_b32 s60, s26
+; SI-NEXT: v_readfirstlane_b32 s47, v30
+; SI-NEXT: v_readfirstlane_b32 s58, v29
+; SI-NEXT: v_readfirstlane_b32 s72, v27
+; SI-NEXT: v_readfirstlane_b32 s73, v26
; SI-NEXT: ; implicit-def: $vgpr47 : SGPR spill to VGPR lane
+; SI-NEXT: s_mov_b32 s57, s21
+; SI-NEXT: s_mov_b32 s89, s20
+; SI-NEXT: s_mov_b32 s61, s27
+; SI-NEXT: s_mov_b32 s45, s28
; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v23
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v1
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:256
+; SI-NEXT: v_readfirstlane_b32 s4, v2
+; SI-NEXT: v_writelane_b32 v56, s4, 8
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:252
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:248
-; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:232
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260
-; SI-NEXT: v_readfirstlane_b32 s4, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:228
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:224
-; SI-NEXT: v_readfirstlane_b32 s51, v19
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:220
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:216
-; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:248
+; SI-NEXT: v_writelane_b32 v56, s4, 9
+; SI-NEXT: v_readfirstlane_b32 s4, v22
+; SI-NEXT: v_writelane_b32 v56, s4, 10
+; SI-NEXT: v_readfirstlane_b32 s4, v19
+; SI-NEXT: v_writelane_b32 v56, s4, 11
+; SI-NEXT: v_readfirstlane_b32 s4, v20
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:220
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:216
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:236
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:232
+; SI-NEXT: v_writelane_b32 v56, s4, 12
+; SI-NEXT: v_readfirstlane_b32 s4, v25
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:212
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:208
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:228
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:224
+; SI-NEXT: v_writelane_b32 v56, s4, 13
+; SI-NEXT: v_readfirstlane_b32 s4, v21
+; SI-NEXT: v_writelane_b32 v56, s4, 14
+; SI-NEXT: v_readfirstlane_b32 s4, v18
+; SI-NEXT: v_writelane_b32 v56, s4, 15
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s8, v22
-; SI-NEXT: v_writelane_b32 v56, s8, 21
-; SI-NEXT: v_readfirstlane_b32 s8, v20
-; SI-NEXT: v_writelane_b32 v56, s8, 22
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:184
-; SI-NEXT: v_readfirstlane_b32 s39, v12
-; SI-NEXT: v_readfirstlane_b32 s31, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
-; SI-NEXT: v_readfirstlane_b32 s5, v24
-; SI-NEXT: v_readfirstlane_b32 s8, v17
-; SI-NEXT: v_writelane_b32 v56, s8, 23
-; SI-NEXT: v_readfirstlane_b32 s8, v14
-; SI-NEXT: v_writelane_b32 v56, s8, 24
-; SI-NEXT: v_readfirstlane_b32 s8, v13
-; SI-NEXT: v_writelane_b32 v56, s8, 25
-; SI-NEXT: v_readfirstlane_b32 s8, v11
-; SI-NEXT: v_writelane_b32 v56, s8, 26
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:156
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:152
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:172
+; SI-NEXT: v_readfirstlane_b32 s4, v11
+; SI-NEXT: v_writelane_b32 v56, s4, 16
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:172
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:168
-; SI-NEXT: v_readfirstlane_b32 s8, v8
-; SI-NEXT: v_readfirstlane_b32 s37, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:104
-; SI-NEXT: v_writelane_b32 v56, s8, 27
-; SI-NEXT: v_readfirstlane_b32 s8, v6
-; SI-NEXT: v_writelane_b32 v56, s8, 28
-; SI-NEXT: v_readfirstlane_b32 s8, v5
-; SI-NEXT: v_writelane_b32 v56, s8, 29
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s8, v4
-; SI-NEXT: v_writelane_b32 v56, s8, 30
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
-; SI-NEXT: v_readfirstlane_b32 s8, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
+; SI-NEXT: v_readfirstlane_b32 s4, v9
+; SI-NEXT: v_writelane_b32 v56, s4, 17
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:164
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
+; SI-NEXT: v_readfirstlane_b32 s26, v15
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v8
+; SI-NEXT: v_readfirstlane_b32 s50, v7
+; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_writelane_b32 v56, s4, 18
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140
; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:136
+; SI-NEXT: v_readfirstlane_b32 s85, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:128
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:148
+; SI-NEXT: v_readfirstlane_b32 s69, v4
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 s4, v3
+; SI-NEXT: v_writelane_b32 v56, s4, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v10
+; SI-NEXT: v_writelane_b32 v56, s4, 20
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_readfirstlane_b32 s4, v13
+; SI-NEXT: v_writelane_b32 v56, s4, 21
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:108
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:148
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:144
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:164
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:180
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:156
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:152
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:180
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:176
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:196
+; SI-NEXT: v_readfirstlane_b32 s93, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:184
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:196
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:192
-; SI-NEXT: v_writelane_b32 v56, s8, 31
-; SI-NEXT: v_readfirstlane_b32 s85, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
-; SI-NEXT: v_readfirstlane_b32 s35, v10
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
-; SI-NEXT: v_readfirstlane_b32 s48, v18
-; SI-NEXT: v_readfirstlane_b32 s63, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
-; SI-NEXT: v_readfirstlane_b32 s38, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:36
-; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
-; SI-NEXT: v_readfirstlane_b32 s30, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:28
-; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
-; SI-NEXT: v_readfirstlane_b32 s82, v1
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
+; SI-NEXT: v_readfirstlane_b32 s4, v14
+; SI-NEXT: v_writelane_b32 v56, s4, 22
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: v_writelane_b32 v56, s4, 23
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:80
-; SI-NEXT: v_readfirstlane_b32 s78, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
-; SI-NEXT: v_readfirstlane_b32 s86, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:12
+; SI-NEXT: v_readfirstlane_b32 s4, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:92
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
+; SI-NEXT: v_writelane_b32 v56, s4, 24
+; SI-NEXT: v_readfirstlane_b32 s4, v12
+; SI-NEXT: v_writelane_b32 v56, s4, 25
+; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v56, s4, 26
+; SI-NEXT: v_readfirstlane_b32 s4, v22
+; SI-NEXT: v_readfirstlane_b32 s70, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:12
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:8
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s61, v28
+; SI-NEXT: v_readfirstlane_b32 s71, v28
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32
-; SI-NEXT: v_readfirstlane_b32 s84, v23
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:20
+; SI-NEXT: v_writelane_b32 v56, s4, 27
+; SI-NEXT: v_readfirstlane_b32 s4, v17
+; SI-NEXT: v_writelane_b32 v56, s4, 28
+; SI-NEXT: v_readfirstlane_b32 s4, v23
+; SI-NEXT: v_writelane_b32 v56, s4, 29
+; SI-NEXT: v_readfirstlane_b32 s4, v19
+; SI-NEXT: v_writelane_b32 v56, s4, 30
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
+; SI-NEXT: v_readfirstlane_b32 s4, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:20
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:16
-; SI-NEXT: v_readfirstlane_b32 s11, v0
-; SI-NEXT: v_readfirstlane_b32 s8, v53
-; SI-NEXT: v_readfirstlane_b32 s24, v12
-; SI-NEXT: v_readfirstlane_b32 s29, v52
-; SI-NEXT: v_readfirstlane_b32 s97, v29
-; SI-NEXT: v_readfirstlane_b32 s98, v20
-; SI-NEXT: v_readfirstlane_b32 s49, v25
-; SI-NEXT: v_readfirstlane_b32 s73, v11
-; SI-NEXT: v_readfirstlane_b32 s96, v33
-; SI-NEXT: v_readfirstlane_b32 s26, v31
-; SI-NEXT: v_readfirstlane_b32 s94, v14
-; SI-NEXT: v_readfirstlane_b32 s72, v7
-; SI-NEXT: v_readfirstlane_b32 s58, v39
-; SI-NEXT: v_readfirstlane_b32 s81, v4
-; SI-NEXT: v_readfirstlane_b32 s60, v37
-; SI-NEXT: v_readfirstlane_b32 s47, v3
-; SI-NEXT: v_readfirstlane_b32 s76, v38
-; SI-NEXT: v_readfirstlane_b32 s91, v5
-; SI-NEXT: v_readfirstlane_b32 s21, v35
-; SI-NEXT: v_readfirstlane_b32 s57, v36
-; SI-NEXT: v_readfirstlane_b32 s87, v8
-; SI-NEXT: v_readfirstlane_b32 s99, v34
-; SI-NEXT: v_readfirstlane_b32 s90, v6
-; SI-NEXT: v_readfirstlane_b32 s28, v13
-; SI-NEXT: v_readfirstlane_b32 s25, v32
-; SI-NEXT: v_readfirstlane_b32 s74, v17
-; SI-NEXT: v_readfirstlane_b32 s93, v30
-; SI-NEXT: v_readfirstlane_b32 s23, v22
-; SI-NEXT: v_readfirstlane_b32 s50, v26
-; SI-NEXT: v_readfirstlane_b32 s95, v27
-; SI-NEXT: v_readfirstlane_b32 s34, v24
-; SI-NEXT: v_readfirstlane_b32 s45, v9
-; SI-NEXT: v_readfirstlane_b32 s15, v49
-; SI-NEXT: v_readfirstlane_b32 s56, v10
-; SI-NEXT: v_readfirstlane_b32 s13, v51
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
+; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_readfirstlane_b32 s19, v5
+; SI-NEXT: v_readfirstlane_b32 s66, v37
+; SI-NEXT: v_readfirstlane_b32 s86, v35
+; SI-NEXT: v_readfirstlane_b32 vcc_hi, v36
+; SI-NEXT: v_readfirstlane_b32 s87, v4
+; SI-NEXT: v_readfirstlane_b32 s36, v6
+; SI-NEXT: v_writelane_b32 v56, s96, 32
+; SI-NEXT: v_readfirstlane_b32 s55, v10
+; SI-NEXT: v_readfirstlane_b32 s5, v13
+; SI-NEXT: v_readfirstlane_b32 s4, v39
+; SI-NEXT: v_readfirstlane_b32 s6, v48
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_readfirstlane_b32 s65, v38
+; SI-NEXT: v_readfirstlane_b32 s99, v7
+; SI-NEXT: v_readfirstlane_b32 s49, v33
+; SI-NEXT: v_readfirstlane_b32 s59, v34
+; SI-NEXT: v_writelane_b32 v56, s95, 33
+; SI-NEXT: v_readfirstlane_b32 s56, v9
+; SI-NEXT: v_writelane_b32 v56, s92, 34
+; SI-NEXT: v_readfirstlane_b32 s30, v32
+; SI-NEXT: v_writelane_b32 v56, s54, 35
+; SI-NEXT: v_readfirstlane_b32 s67, v31
+; SI-NEXT: v_writelane_b32 v56, s38, 36
+; SI-NEXT: v_readfirstlane_b32 s21, v8
+; SI-NEXT: v_writelane_b32 v56, s34, 37
+; SI-NEXT: v_readfirstlane_b32 s16, v15
+; SI-NEXT: v_writelane_b32 v56, s39, 38
+; SI-NEXT: v_readfirstlane_b32 s20, v30
+; SI-NEXT: v_writelane_b32 v56, s90, 39
+; SI-NEXT: v_readfirstlane_b32 s27, v11
+; SI-NEXT: v_writelane_b32 v56, s37, 40
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s9, v15
-; SI-NEXT: v_readfirstlane_b32 s69, v55
-; SI-NEXT: v_readfirstlane_b32 s68, v21
-; SI-NEXT: v_writelane_b32 v47, s68, 0
-; SI-NEXT: v_readfirstlane_b32 s70, v40
-; SI-NEXT: v_writelane_b32 v47, s69, 1
+; SI-NEXT: v_readfirstlane_b32 s17, v29
+; SI-NEXT: v_writelane_b32 v56, s77, 41
+; SI-NEXT: v_readfirstlane_b32 s98, v18
+; SI-NEXT: v_writelane_b32 v56, s48, 42
+; SI-NEXT: v_readfirstlane_b32 s28, v27
+; SI-NEXT: v_writelane_b32 v56, s63, 43
+; SI-NEXT: v_readfirstlane_b32 s68, v16
+; SI-NEXT: v_writelane_b32 v56, s80, 44
+; SI-NEXT: v_readfirstlane_b32 s84, v26
+; SI-NEXT: v_writelane_b32 v56, s94, 45
+; SI-NEXT: v_writelane_b32 v56, s35, 46
+; SI-NEXT: v_writelane_b32 v56, s79, 47
+; SI-NEXT: v_readfirstlane_b32 s81, v25
+; SI-NEXT: v_writelane_b32 v56, s91, 48
+; SI-NEXT: v_readfirstlane_b32 s82, v21
+; SI-NEXT: v_writelane_b32 v56, s75, 49
+; SI-NEXT: v_writelane_b32 v56, s31, 50
+; SI-NEXT: v_writelane_b32 v56, s62, 51
+; SI-NEXT: v_writelane_b32 v56, s72, 52
+; SI-NEXT: v_readfirstlane_b32 s10, v50
+; SI-NEXT: v_readfirstlane_b32 s12, v2
+; SI-NEXT: v_writelane_b32 v47, s12, 0
+; SI-NEXT: v_writelane_b32 v47, s10, 1
+; SI-NEXT: v_writelane_b32 v47, s5, 2
+; SI-NEXT: v_writelane_b32 v47, s4, 3
+; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v47, s6, 4
+; SI-NEXT: v_writelane_b32 v47, s13, 5
+; SI-NEXT: v_writelane_b32 v47, s64, 6
+; SI-NEXT: v_writelane_b32 v47, s66, 7
+; SI-NEXT: v_readfirstlane_b32 s15, v43
+; SI-NEXT: v_readfirstlane_b32 s41, v28
+; SI-NEXT: v_writelane_b32 v47, s41, 8
+; SI-NEXT: v_readfirstlane_b32 s40, v44
+; SI-NEXT: v_writelane_b32 v47, s15, 9
+; SI-NEXT: v_writelane_b32 v47, s40, 10
+; SI-NEXT: v_writelane_b32 v47, s65, 11
+; SI-NEXT: v_readfirstlane_b32 s46, v20
+; SI-NEXT: v_writelane_b32 v56, s58, 53
; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s71, v16
-; SI-NEXT: v_writelane_b32 v47, s70, 2
+; SI-NEXT: v_readfirstlane_b32 s88, v19
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_readfirstlane_b32 s78, v41
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s80, v18
-; SI-NEXT: v_writelane_b32 v47, s71, 3
-; SI-NEXT: v_writelane_b32 v47, s80, 4
-; SI-NEXT: v_writelane_b32 v47, s8, 5
-; SI-NEXT: v_writelane_b32 v47, s9, 6
-; SI-NEXT: v_writelane_b32 v47, s24, 7
-; SI-NEXT: v_writelane_b32 v47, s13, 8
-; SI-NEXT: v_writelane_b32 v47, s29, 9
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s14, v1
-; SI-NEXT: v_writelane_b32 v47, s11, 10
-; SI-NEXT: v_writelane_b32 v47, s14, 11
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s40, v50
-; SI-NEXT: v_writelane_b32 v47, s15, 12
-; SI-NEXT: v_writelane_b32 v47, s40, 13
+; SI-NEXT: v_readfirstlane_b32 s9, v24
+; SI-NEXT: v_writelane_b32 v47, s9, 12
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_readfirstlane_b32 s42, v42
+; SI-NEXT: v_writelane_b32 v47, s78, 13
+; SI-NEXT: v_writelane_b32 v47, s42, 14
+; SI-NEXT: v_writelane_b32 v47, s55, 15
+; SI-NEXT: v_writelane_b32 v47, s19, 16
+; SI-NEXT: v_writelane_b32 v47, s86, 17
+; SI-NEXT: v_writelane_b32 v47, vcc_hi, 18
+; SI-NEXT: v_writelane_b32 v47, s87, 19
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_readfirstlane_b32 s42, v2
-; SI-NEXT: v_writelane_b32 v47, s56, 14
-; SI-NEXT: v_writelane_b32 v47, s42, 15
+; SI-NEXT: v_readfirstlane_b32 s97, v23
+; SI-NEXT: v_writelane_b32 v47, s46, 20
+; SI-NEXT: v_readfirstlane_b32 s83, v55
+; SI-NEXT: v_writelane_b32 v47, s97, 21
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s43, v48
-; SI-NEXT: v_writelane_b32 v47, s58, 16
-; SI-NEXT: v_writelane_b32 v47, s43, 17
-; SI-NEXT: v_writelane_b32 v47, s45, 18
-; SI-NEXT: v_writelane_b32 v47, s47, 19
-; SI-NEXT: v_writelane_b32 v47, s60, 20
-; SI-NEXT: v_writelane_b32 v47, s76, 21
-; SI-NEXT: v_writelane_b32 v47, s72, 22
-; SI-NEXT: v_writelane_b32 v47, s91, 23
-; SI-NEXT: v_writelane_b32 v47, s21, 24
-; SI-NEXT: v_writelane_b32 v47, s57, 25
-; SI-NEXT: v_readfirstlane_b32 s41, v54
-; SI-NEXT: v_writelane_b32 v47, s81, 26
-; SI-NEXT: v_writelane_b32 v56, s41, 32
-; SI-NEXT: v_writelane_b32 v47, s87, 27
-; SI-NEXT: v_writelane_b32 v56, s18, 33
-; SI-NEXT: v_writelane_b32 v47, s96, 28
-; SI-NEXT: v_writelane_b32 v56, s6, 34
-; SI-NEXT: v_writelane_b32 v47, s99, 29
-; SI-NEXT: v_writelane_b32 v56, s17, 35
-; SI-NEXT: v_writelane_b32 v47, s90, 30
-; SI-NEXT: v_writelane_b32 v56, s16, 36
-; SI-NEXT: v_writelane_b32 v47, s28, 31
-; SI-NEXT: v_writelane_b32 v56, s7, 37
-; SI-NEXT: v_writelane_b32 v47, s26, 32
-; SI-NEXT: v_writelane_b32 v56, s92, 38
-; SI-NEXT: v_writelane_b32 v47, s37, 33
-; SI-NEXT: v_writelane_b32 v56, s44, 39
-; SI-NEXT: v_writelane_b32 v47, s25, 34
-; SI-NEXT: v_writelane_b32 v56, s59, 40
-; SI-NEXT: v_writelane_b32 v47, s78, 35
-; SI-NEXT: v_writelane_b32 v56, s27, 41
-; SI-NEXT: v_writelane_b32 v47, s73, 36
-; SI-NEXT: v_writelane_b32 v56, s62, 42
-; SI-NEXT: v_writelane_b32 v47, s74, 37
-; SI-NEXT: v_writelane_b32 v56, s79, 43
-; SI-NEXT: v_writelane_b32 v47, s97, 38
-; SI-NEXT: v_writelane_b32 v56, s22, 44
-; SI-NEXT: v_writelane_b32 v47, s35, 39
-; SI-NEXT: v_writelane_b32 v56, s75, 45
-; SI-NEXT: v_writelane_b32 v47, s39, 40
-; SI-NEXT: v_writelane_b32 v56, s36, 46
-; SI-NEXT: v_writelane_b32 v47, s85, 41
-; SI-NEXT: v_writelane_b32 v56, s19, 47
-; SI-NEXT: v_writelane_b32 v47, s93, 42
-; SI-NEXT: v_writelane_b32 v56, s77, 48
-; SI-NEXT: v_writelane_b32 v47, s63, 43
-; SI-NEXT: v_writelane_b32 v56, s88, 49
-; SI-NEXT: v_writelane_b32 v47, s86, 44
-; SI-NEXT: v_writelane_b32 v56, s12, 50
-; SI-NEXT: v_writelane_b32 v47, s94, 45
-; SI-NEXT: v_writelane_b32 v56, s89, 51
-; SI-NEXT: v_writelane_b32 v47, s38, 46
-; SI-NEXT: v_writelane_b32 v56, s46, 52
-; SI-NEXT: v_writelane_b32 v47, s61, 47
-; SI-NEXT: v_writelane_b32 v56, s20, 53
-; SI-NEXT: v_writelane_b32 v47, s23, 48
-; SI-NEXT: v_writelane_b32 v56, vcc_lo, 54
-; SI-NEXT: v_writelane_b32 v47, s50, 49
-; SI-NEXT: v_writelane_b32 v56, vcc_hi, 55
-; SI-NEXT: v_writelane_b32 v47, s82, 50
+; SI-NEXT: v_readfirstlane_b32 s76, v40
+; SI-NEXT: v_writelane_b32 v47, s83, 22
+; SI-NEXT: v_writelane_b32 v47, s76, 23
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s53, v28
-; SI-NEXT: v_writelane_b32 v56, s52, 56
-; SI-NEXT: v_writelane_b32 v47, s31, 51
-; SI-NEXT: v_readfirstlane_b32 s54, v43
-; SI-NEXT: v_writelane_b32 v56, s53, 57
-; SI-NEXT: v_writelane_b32 v47, s95, 52
+; SI-NEXT: v_readfirstlane_b32 s53, v22
+; SI-NEXT: v_writelane_b32 v47, s88, 24
+; SI-NEXT: v_readfirstlane_b32 s74, v53
+; SI-NEXT: v_writelane_b32 v47, s53, 25
+; SI-NEXT: v_writelane_b32 v47, s74, 26
+; SI-NEXT: v_writelane_b32 v47, s99, 27
+; SI-NEXT: v_writelane_b32 v47, s36, 28
+; SI-NEXT: v_writelane_b32 v47, s49, 29
+; SI-NEXT: v_writelane_b32 v47, s59, 30
+; SI-NEXT: v_writelane_b32 v47, s56, 31
+; SI-NEXT: v_writelane_b32 v47, s30, 32
+; SI-NEXT: v_writelane_b32 v47, s67, 33
+; SI-NEXT: v_writelane_b32 v47, s21, 34
+; SI-NEXT: v_writelane_b32 v47, s16, 35
+; SI-NEXT: v_writelane_b32 v47, s20, 36
+; SI-NEXT: v_writelane_b32 v47, s27, 37
+; SI-NEXT: v_writelane_b32 v47, s17, 38
+; SI-NEXT: v_writelane_b32 v47, s98, 39
+; SI-NEXT: v_writelane_b32 v47, s28, 40
+; SI-NEXT: v_writelane_b32 v47, s68, 41
+; SI-NEXT: v_writelane_b32 v47, s84, 42
+; SI-NEXT: v_writelane_b32 v47, s71, 43
+; SI-NEXT: v_writelane_b32 v47, s70, 44
+; SI-NEXT: v_writelane_b32 v47, s81, 45
+; SI-NEXT: v_writelane_b32 v47, s82, 46
+; SI-NEXT: v_writelane_b32 v47, s85, 47
+; SI-NEXT: v_writelane_b32 v47, s69, 48
+; SI-NEXT: v_writelane_b32 v47, s18, 49
+; SI-NEXT: v_writelane_b32 v56, s73, 54
+; SI-NEXT: v_writelane_b32 v47, s22, 50
+; SI-NEXT: v_readfirstlane_b32 s52, v17
+; SI-NEXT: v_writelane_b32 v56, s47, 55
+; SI-NEXT: v_writelane_b32 v47, s23, 51
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_readfirstlane_b32 s55, v44
-; SI-NEXT: v_writelane_b32 v56, s54, 58
-; SI-NEXT: v_writelane_b32 v47, s30, 53
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s64, v23
-; SI-NEXT: v_writelane_b32 v56, s55, 59
-; SI-NEXT: v_writelane_b32 v47, s98, 54
-; SI-NEXT: v_readfirstlane_b32 s65, v41
-; SI-NEXT: v_writelane_b32 v56, s64, 60
-; SI-NEXT: v_writelane_b32 v47, s48, 55
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_readfirstlane_b32 s66, v42
-; SI-NEXT: v_writelane_b32 v56, s65, 61
-; SI-NEXT: v_writelane_b32 v47, s84, 56
-; SI-NEXT: v_readfirstlane_b32 s67, v19
-; SI-NEXT: v_writelane_b32 v56, s66, 62
-; SI-NEXT: v_writelane_b32 v47, s49, 57
-; SI-NEXT: v_writelane_b32 v56, s67, 63
-; SI-NEXT: v_writelane_b32 v47, s34, 58
+; SI-NEXT: v_readfirstlane_b32 s51, v54
+; SI-NEXT: v_writelane_b32 v56, s52, 56
+; SI-NEXT: v_writelane_b32 v47, s57, 52
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_readfirstlane_b32 s7, v12
+; SI-NEXT: v_writelane_b32 v56, s51, 57
+; SI-NEXT: v_writelane_b32 v47, s89, 53
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_readfirstlane_b32 s8, v52
+; SI-NEXT: v_writelane_b32 v56, s7, 58
+; SI-NEXT: v_writelane_b32 v47, s60, 54
+; SI-NEXT: v_readfirstlane_b32 s44, v51
+; SI-NEXT: v_writelane_b32 v56, s8, 59
+; SI-NEXT: v_writelane_b32 v47, s61, 55
+; SI-NEXT: v_readfirstlane_b32 s11, v0
+; SI-NEXT: v_writelane_b32 v56, s44, 60
+; SI-NEXT: v_writelane_b32 v47, s25, 56
+; SI-NEXT: v_readfirstlane_b32 s43, v1
+; SI-NEXT: v_writelane_b32 v56, s11, 61
+; SI-NEXT: v_writelane_b32 v47, s24, 57
+; SI-NEXT: v_readfirstlane_b32 s14, v49
+; SI-NEXT: v_writelane_b32 v56, s43, 62
+; SI-NEXT: v_writelane_b32 v47, s29, 58
+; SI-NEXT: v_writelane_b32 v56, s14, 63
+; SI-NEXT: v_writelane_b32 v47, s45, 59
; SI-NEXT: s_cbranch_scc0 .LBB89_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_writelane_b32 v46, s83, 7
-; SI-NEXT: v_writelane_b32 v46, s4, 8
-; SI-NEXT: s_and_b32 s4, s16, 0xff
+; SI-NEXT: v_writelane_b32 v46, s50, 19
+; SI-NEXT: s_mov_b32 s50, s35
+; SI-NEXT: s_mov_b32 s35, s77
+; SI-NEXT: s_mov_b32 s77, s45
+; SI-NEXT: s_mov_b32 s45, s29
+; SI-NEXT: v_readlane_b32 s29, v56, 2
+; SI-NEXT: s_mov_b32 s69, s80
+; SI-NEXT: s_and_b32 s80, s29, 0xff
+; SI-NEXT: s_mov_b32 s85, s47
+; SI-NEXT: s_lshl_b32 s47, s80, 16
+; SI-NEXT: s_mov_b32 s80, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s58
+; SI-NEXT: v_readlane_b32 s58, v56, 1
+; SI-NEXT: s_lshl_b32 s58, s58, 24
+; SI-NEXT: v_writelane_b32 v46, s93, 20
+; SI-NEXT: v_writelane_b32 v46, s26, 21
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s18, 0xff
+; SI-NEXT: v_readlane_b32 s26, v56, 0
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s26, 24
+; SI-NEXT: v_writelane_b32 v47, s93, 60
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s89, 0xff
+; SI-NEXT: s_lshl_b32 s58, s57, 8
+; SI-NEXT: s_or_b32 s18, s47, s58
+; SI-NEXT: s_and_b32 s47, s22, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s23, 24
+; SI-NEXT: v_writelane_b32 v47, s93, 61
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s24, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s25, 24
+; SI-NEXT: v_writelane_b32 v47, s93, 63
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s60, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s61, 24
+; SI-NEXT: v_writelane_b32 v46, s18, 22
+; SI-NEXT: v_writelane_b32 v47, s93, 62
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s77, 0xff
+; SI-NEXT: s_lshl_b32 s58, s45, 8
+; SI-NEXT: v_writelane_b32 v46, s93, 0
+; SI-NEXT: s_or_b32 s93, s47, s58
+; SI-NEXT: s_and_b32 s47, s90, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 9
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s18, 24
+; SI-NEXT: v_readlane_b32 s18, v56, 8
+; SI-NEXT: s_or_b32 s90, s58, s47
+; SI-NEXT: s_and_b32 s47, s18, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 7
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s18, 24
+; SI-NEXT: v_readlane_b32 s18, v56, 6
+; SI-NEXT: s_or_b32 s89, s58, s47
+; SI-NEXT: s_and_b32 s47, s18, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 5
+; SI-NEXT: s_lshl_b32 s58, s18, 24
+; SI-NEXT: v_readlane_b32 s18, v56, 4
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_and_b32 s45, s18, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 3
+; SI-NEXT: s_or_b32 s61, s58, s47
+; SI-NEXT: s_lshl_b32 s47, s18, 8
+; SI-NEXT: v_writelane_b32 v46, s90, 2
+; SI-NEXT: s_or_b32 s90, s45, s47
+; SI-NEXT: s_and_b32 s47, s95, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s96, 24
+; SI-NEXT: v_writelane_b32 v46, s89, 1
+; SI-NEXT: s_or_b32 s60, s58, s47
+; SI-NEXT: s_and_b32 s47, s34, 0xff
+; SI-NEXT: v_writelane_b32 v46, s61, 3
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s54, 24
+; SI-NEXT: v_writelane_b32 v46, s60, 5
+; SI-NEXT: s_or_b32 s57, s57, s47
+; SI-NEXT: s_and_b32 s47, s92, 0xff
+; SI-NEXT: v_writelane_b32 v46, s57, 4
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s38, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 6
+; SI-NEXT: s_and_b32 s47, s48, 0xff
+; SI-NEXT: s_lshl_b32 s57, s37, 8
+; SI-NEXT: s_or_b32 s58, s47, s57
+; SI-NEXT: s_and_b32 s47, s39, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s35, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 7
+; SI-NEXT: s_and_b32 s47, s50, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s94, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 14
+; SI-NEXT: s_and_b32 s47, s63, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s69, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 13
+; SI-NEXT: s_and_b32 s47, s31, 0xff
+; SI-NEXT: s_lshl_b32 s57, s91, 8
+; SI-NEXT: s_or_b32 s45, s47, s57
+; SI-NEXT: s_and_b32 s47, s79, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s75, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 15
+; SI-NEXT: s_and_b32 s47, s73, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s72, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 17
+; SI-NEXT: s_and_b32 s47, s62, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, vcc_lo, 24
+; SI-NEXT: s_and_b32 s41, s41, 0xff
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_lshl_b32 s15, s15, 24
+; SI-NEXT: v_writelane_b32 v46, s47, 16
+; SI-NEXT: s_or_b32 s15, s15, s41
+; SI-NEXT: v_writelane_b32 v46, s15, 18
+; SI-NEXT: s_and_b32 s15, s46, 0xff
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_lshl_b32 s41, s42, 24
+; SI-NEXT: s_or_b32 s15, s41, s15
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: v_writelane_b32 v46, s15, 8
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s15, s78, 24
+; SI-NEXT: s_or_b32 s9, s15, s9
+; SI-NEXT: v_writelane_b32 v46, s9, 9
+; SI-NEXT: s_and_b32 s9, s88, 0xff
+; SI-NEXT: s_lshl_b32 s15, s76, 8
+; SI-NEXT: s_or_b32 s15, s9, s15
+; SI-NEXT: s_and_b32 s9, s97, 0xff
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s41, s83, 24
+; SI-NEXT: s_or_b32 s9, s41, s9
+; SI-NEXT: v_writelane_b32 v46, s9, 11
+; SI-NEXT: s_and_b32 s9, s52, 0xff
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s41, s51, 24
+; SI-NEXT: s_or_b32 s9, s41, s9
+; SI-NEXT: v_writelane_b32 v46, s9, 10
+; SI-NEXT: s_and_b32 s9, s53, 0xff
+; SI-NEXT: s_and_b32 s47, s85, 0xff
+; SI-NEXT: s_lshl_b32 s40, s40, 8
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s41, s74, 24
+; SI-NEXT: s_or_b32 s40, s47, s40
+; SI-NEXT: s_mov_b32 s47, s49
+; SI-NEXT: s_or_b32 s49, s41, s9
+; SI-NEXT: s_and_b32 s9, s11, 0xff
+; SI-NEXT: s_lshl_b32 s8, s8, 8
+; SI-NEXT: s_and_b32 s7, s7, 0xff
+; SI-NEXT: s_and_b32 s5, s5, 0xff
+; SI-NEXT: s_or_b32 s51, s9, s8
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s8, s44, 24
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 24
+; SI-NEXT: s_or_b32 s7, s8, s7
+; SI-NEXT: s_or_b32 s11, s4, s5
+; SI-NEXT: s_and_b32 s4, s55, 0xff
+; SI-NEXT: v_writelane_b32 v46, s7, 12
+; SI-NEXT: s_and_b32 s7, s12, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_mov_b32 s83, s51
-; SI-NEXT: s_mov_b32 s51, s10
-; SI-NEXT: s_mov_b32 s10, s5
-; SI-NEXT: s_lshl_b32 s5, s17, 24
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_writelane_b32 v47, s4, 59
-; SI-NEXT: s_and_b32 s4, s18, 0xff
+; SI-NEXT: s_lshl_b32 s5, s65, 24
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s8, s10, 24
+; SI-NEXT: s_or_b32 s55, s5, s4
+; SI-NEXT: s_and_b32 s4, s64, 0xff
+; SI-NEXT: s_mov_b32 s42, s16
+; SI-NEXT: s_or_b32 s16, s8, s7
+; SI-NEXT: s_and_b32 s7, s43, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s5, s6, 24
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v56, 8
-; SI-NEXT: v_writelane_b32 v47, s4, 60
-; SI-NEXT: s_and_b32 s4, s7, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v46, s4, 9
-; SI-NEXT: v_readlane_b32 s4, v56, 7
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 6
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s4, 24
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s4, v56, 5
-; SI-NEXT: v_writelane_b32 v47, s5, 62
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 4
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s4, 24
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s4, v56, 3
-; SI-NEXT: v_writelane_b32 v47, s5, 61
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 2
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s4, 24
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s4, v56, 1
-; SI-NEXT: v_writelane_b32 v47, s5, 63
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 0
-; SI-NEXT: s_lshl_b32 s6, s4, 8
-; SI-NEXT: s_or_b32 s4, s5, s6
-; SI-NEXT: s_and_b32 s6, s52, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 24
-; SI-NEXT: v_writelane_b32 v46, s4, 10
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 1
-; SI-NEXT: s_and_b32 s6, vcc_hi, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s20, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 0
-; SI-NEXT: s_and_b32 s6, s36, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s22, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 2
-; SI-NEXT: s_and_b32 s6, s46, 0xff
-; SI-NEXT: s_lshl_b32 s7, s12, 8
-; SI-NEXT: s_or_b32 s4, s6, s7
-; SI-NEXT: s_and_b32 s6, s79, 0xff
+; SI-NEXT: s_lshl_b32 s5, s66, 24
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s8, s14, 24
+; SI-NEXT: s_or_b32 s22, s5, s4
+; SI-NEXT: s_and_b32 s4, s87, 0xff
+; SI-NEXT: s_lshl_b32 s5, vcc_hi, 8
+; SI-NEXT: s_or_b32 s7, s8, s7
+; SI-NEXT: s_and_b32 s8, s13, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s5, s4, s5
+; SI-NEXT: s_and_b32 s4, s19, 0xff
+; SI-NEXT: s_or_b32 s10, s8, s6
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s86, 24
+; SI-NEXT: s_or_b32 s64, s6, s4
+; SI-NEXT: s_and_b32 s4, s36, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s59, 24
+; SI-NEXT: s_or_b32 s65, s6, s4
+; SI-NEXT: s_and_b32 s4, s99, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s47, 24
+; SI-NEXT: s_or_b32 s94, s6, s4
+; SI-NEXT: s_and_b32 s4, s21, 0xff
+; SI-NEXT: s_lshl_b32 s6, s30, 8
+; SI-NEXT: s_or_b32 s29, s4, s6
+; SI-NEXT: s_and_b32 s4, s56, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s67, 24
+; SI-NEXT: s_or_b32 s92, s6, s4
+; SI-NEXT: s_and_b32 s4, s27, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s20, 24
+; SI-NEXT: s_or_b32 s88, s6, s4
+; SI-NEXT: s_and_b32 s4, s42, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s17, 24
+; SI-NEXT: s_or_b32 s89, s6, s4
+; SI-NEXT: s_and_b32 s4, s68, 0xff
+; SI-NEXT: s_lshl_b32 s6, s28, 8
+; SI-NEXT: s_or_b32 s23, s4, s6
+; SI-NEXT: s_and_b32 s4, s98, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s84, 24
+; SI-NEXT: s_or_b32 s78, s6, s4
+; SI-NEXT: s_and_b32 s4, s82, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s81, 24
+; SI-NEXT: s_or_b32 s62, s6, s4
+; SI-NEXT: s_and_b32 s4, s71, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s70, 24
+; SI-NEXT: s_or_b32 s76, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 31
+; SI-NEXT: v_readlane_b32 s6, v56, 30
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s24, s4, s6
+; SI-NEXT: v_readlane_b32 s4, v56, 29
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 28
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s61, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 27
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 26
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s47, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 25
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 24
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s57, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 23
+; SI-NEXT: v_readlane_b32 s6, v56, 22
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s25, s4, s6
+; SI-NEXT: v_readlane_b32 s4, v56, 21
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 20
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s46, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 19
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v47, 48
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s14, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v47, 47
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s4, 16
+; SI-NEXT: v_readlane_b32 s4, v56, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 24
+; SI-NEXT: s_or_b32 s44, s4, s6
+; SI-NEXT: v_readlane_b32 s4, v46, 19
+; SI-NEXT: s_mov_b32 s75, s4
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s80, 8
+; SI-NEXT: s_or_b32 s4, s4, s6
+; SI-NEXT: v_readlane_b32 s6, v56, 17
+; SI-NEXT: v_readlane_b32 s77, v56, 16
+; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s62, 24
-; SI-NEXT: v_writelane_b32 v46, s4, 11
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 4
-; SI-NEXT: s_and_b32 s6, s89, 0xff
+; SI-NEXT: s_lshl_b32 s8, s77, 24
+; SI-NEXT: s_or_b32 s12, s8, s6
+; SI-NEXT: v_readlane_b32 s6, v46, 21
+; SI-NEXT: s_mov_b32 s20, s6
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: v_readlane_b32 s8, v46, 20
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s77, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 3
-; SI-NEXT: s_and_b32 s6, s27, 0xff
+; SI-NEXT: s_mov_b32 s91, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 24
+; SI-NEXT: s_or_b32 s8, s8, s6
+; SI-NEXT: v_readlane_b32 s6, v56, 15
+; SI-NEXT: s_mov_b32 s72, s6
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: v_readlane_b32 s9, v56, 14
+; SI-NEXT: v_readlane_b32 s60, v56, 12
+; SI-NEXT: v_readlane_b32 s18, v56, 13
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s59, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 5
-; SI-NEXT: s_and_b32 s6, s88, 0xff
-; SI-NEXT: s_lshl_b32 s7, s75, 8
-; SI-NEXT: s_or_b32 s4, s6, s7
-; SI-NEXT: s_and_b32 s6, s44, 0xff
+; SI-NEXT: s_mov_b32 s17, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 24
+; SI-NEXT: v_readlane_b32 s63, v56, 11
+; SI-NEXT: s_or_b32 s9, s9, s6
+; SI-NEXT: s_and_b32 s6, s18, 0xff
+; SI-NEXT: s_lshl_b32 s13, s60, 8
+; SI-NEXT: s_or_b32 s26, s6, s13
+; SI-NEXT: s_and_b32 s6, s63, 0xff
+; SI-NEXT: v_readlane_b32 s13, v56, 10
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s92, 24
-; SI-NEXT: v_writelane_b32 v46, s4, 12
-; SI-NEXT: s_or_b32 s4, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s4, 6
-; SI-NEXT: s_and_b32 s7, s19, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 20
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s12, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 19
-; SI-NEXT: s_or_b32 s7, s12, s7
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 18
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s18, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 17
-; SI-NEXT: s_or_b32 s18, s18, s12
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 16
-; SI-NEXT: s_lshl_b32 s19, s4, 8
-; SI-NEXT: v_readlane_b32 s4, v56, 15
-; SI-NEXT: s_or_b32 s6, s12, s19
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 14
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s19, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 13
-; SI-NEXT: s_or_b32 s16, s19, s12
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 12
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s20, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 11
-; SI-NEXT: s_or_b32 s5, s20, s12
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 10
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s20, s4, 24
-; SI-NEXT: s_or_b32 s4, s20, s12
-; SI-NEXT: v_readlane_b32 s12, v56, 9
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_lshl_b32 s27, s55, 8
-; SI-NEXT: s_or_b32 s52, s12, s27
-; SI-NEXT: s_and_b32 s12, s53, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s27, s54, 24
-; SI-NEXT: s_or_b32 s17, s27, s12
-; SI-NEXT: s_and_b32 s12, s67, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s27, s66, 24
-; SI-NEXT: s_or_b32 s19, s27, s12
-; SI-NEXT: s_and_b32 s12, s64, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s44, s65, 24
-; SI-NEXT: s_or_b32 s12, s44, s12
-; SI-NEXT: s_and_b32 s44, s71, 0xff
-; SI-NEXT: s_lshl_b32 s46, s70, 8
-; SI-NEXT: s_or_b32 s53, s44, s46
-; SI-NEXT: s_and_b32 s44, s68, 0xff
-; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_lshl_b32 s46, s69, 24
-; SI-NEXT: s_or_b32 s44, s46, s44
-; SI-NEXT: s_and_b32 s46, s9, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s41, s41, 24
-; SI-NEXT: s_or_b32 s41, s41, s46
-; SI-NEXT: s_and_b32 s46, s80, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s59, s8, 24
-; SI-NEXT: s_or_b32 s59, s59, s46
-; SI-NEXT: s_and_b32 s46, s11, 0xff
-; SI-NEXT: s_lshl_b32 s62, s29, 8
-; SI-NEXT: s_or_b32 s54, s46, s62
-; SI-NEXT: s_and_b32 s46, s24, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s62, s13, 24
-; SI-NEXT: s_or_b32 s27, s62, s46
-; SI-NEXT: s_and_b32 s46, s56, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s62, s40, 24
-; SI-NEXT: s_or_b32 s62, s62, s46
-; SI-NEXT: s_and_b32 s46, s14, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s75, s15, 24
-; SI-NEXT: s_or_b32 s75, s75, s46
-; SI-NEXT: s_and_b32 s46, s45, 0xff
-; SI-NEXT: s_lshl_b32 s77, s43, 8
-; SI-NEXT: s_or_b32 s55, s46, s77
-; SI-NEXT: s_and_b32 s46, s42, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s77, s58, 24
-; SI-NEXT: s_or_b32 s29, s77, s46
-; SI-NEXT: s_and_b32 s46, s72, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s79, s76, 24
-; SI-NEXT: s_or_b32 s46, s79, s46
-; SI-NEXT: s_and_b32 s79, s47, 0xff
-; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s88, s60, 24
-; SI-NEXT: s_or_b32 s24, s88, s79
-; SI-NEXT: s_and_b32 s88, s81, 0xff
-; SI-NEXT: s_lshl_b32 s89, s57, 8
-; SI-NEXT: s_mov_b32 s42, s16
-; SI-NEXT: s_or_b32 s16, s88, s89
-; SI-NEXT: s_and_b32 s88, s91, 0xff
-; SI-NEXT: s_lshl_b32 s88, s88, 16
-; SI-NEXT: s_lshl_b32 s89, s21, 24
-; SI-NEXT: s_or_b32 s21, s89, s88
-; SI-NEXT: s_and_b32 s88, s90, 0xff
-; SI-NEXT: s_lshl_b32 s88, s88, 16
-; SI-NEXT: s_lshl_b32 s90, s99, 24
-; SI-NEXT: s_or_b32 s88, s90, s88
-; SI-NEXT: s_and_b32 s90, s87, 0xff
-; SI-NEXT: s_lshl_b32 s90, s90, 16
-; SI-NEXT: s_lshl_b32 s91, s96, 24
-; SI-NEXT: s_or_b32 s40, s91, s90
-; SI-NEXT: s_and_b32 s91, s73, 0xff
-; SI-NEXT: s_lshl_b32 s92, s25, 8
-; SI-NEXT: s_mov_b32 s45, s19
-; SI-NEXT: s_or_b32 s19, s91, s92
-; SI-NEXT: s_and_b32 s91, s28, 0xff
-; SI-NEXT: s_lshl_b32 s91, s91, 16
-; SI-NEXT: s_lshl_b32 s92, s26, 24
-; SI-NEXT: s_or_b32 s92, s92, s91
-; SI-NEXT: s_and_b32 s91, s94, 0xff
-; SI-NEXT: s_lshl_b32 s91, s91, 16
-; SI-NEXT: s_lshl_b32 s93, s93, 24
-; SI-NEXT: s_or_b32 s91, s93, s91
-; SI-NEXT: s_and_b32 s93, s74, 0xff
-; SI-NEXT: s_lshl_b32 s93, s93, 16
-; SI-NEXT: s_lshl_b32 s94, s97, 24
-; SI-NEXT: s_or_b32 s93, s94, s93
-; SI-NEXT: s_and_b32 s94, s98, 0xff
-; SI-NEXT: s_lshl_b32 s95, s95, 8
-; SI-NEXT: s_or_b32 s22, s94, s95
-; SI-NEXT: s_and_b32 s94, s23, 0xff
-; SI-NEXT: s_lshl_b32 s94, s94, 16
-; SI-NEXT: s_lshl_b32 s95, s50, 24
-; SI-NEXT: s_or_b32 s95, s95, s94
-; SI-NEXT: s_and_b32 s94, s34, 0xff
-; SI-NEXT: s_lshl_b32 s94, s94, 16
-; SI-NEXT: s_lshl_b32 vcc_lo, s49, 24
-; SI-NEXT: s_or_b32 s94, vcc_lo, s94
-; SI-NEXT: s_and_b32 vcc_lo, s61, 0xff
-; SI-NEXT: s_lshl_b32 vcc_lo, vcc_lo, 16
-; SI-NEXT: s_lshl_b32 vcc_hi, s86, 24
-; SI-NEXT: s_or_b32 vcc_lo, vcc_hi, vcc_lo
-; SI-NEXT: s_and_b32 vcc_hi, s84, 0xff
-; SI-NEXT: s_lshl_b32 s30, s30, 8
-; SI-NEXT: s_mov_b32 s43, s17
-; SI-NEXT: s_or_b32 s17, vcc_hi, s30
-; SI-NEXT: s_and_b32 vcc_hi, s38, 0xff
-; SI-NEXT: s_lshl_b32 vcc_hi, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s30, s63, 24
-; SI-NEXT: s_or_b32 s30, s30, vcc_hi
-; SI-NEXT: s_and_b32 vcc_hi, s48, 0xff
-; SI-NEXT: s_lshl_b32 vcc_hi, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s31, s31, 24
-; SI-NEXT: s_or_b32 vcc_hi, s31, vcc_hi
-; SI-NEXT: s_and_b32 s31, s39, 0xff
-; SI-NEXT: s_lshl_b32 s31, s31, 16
-; SI-NEXT: s_lshl_b32 s34, s35, 24
-; SI-NEXT: s_or_b32 s31, s34, s31
-; SI-NEXT: s_and_b32 s34, s82, 0xff
-; SI-NEXT: s_lshl_b32 s35, s85, 8
-; SI-NEXT: s_or_b32 s20, s34, s35
-; SI-NEXT: s_and_b32 s34, s78, 0xff
-; SI-NEXT: s_lshl_b32 s34, s34, 16
-; SI-NEXT: s_lshl_b32 s35, s37, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 31
-; SI-NEXT: s_or_b32 s35, s35, s34
-; SI-NEXT: s_and_b32 s34, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 30
-; SI-NEXT: s_lshl_b32 s34, s34, 16
-; SI-NEXT: s_lshl_b32 s36, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 29
-; SI-NEXT: s_or_b32 s34, s36, s34
-; SI-NEXT: s_and_b32 s36, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 28
-; SI-NEXT: s_lshl_b32 s36, s36, 16
-; SI-NEXT: s_lshl_b32 s37, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 27
-; SI-NEXT: s_or_b32 s36, s37, s36
-; SI-NEXT: s_and_b32 s37, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 26
-; SI-NEXT: s_lshl_b32 s38, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v56, 25
-; SI-NEXT: s_or_b32 s23, s37, s38
-; SI-NEXT: s_and_b32 s37, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 24
-; SI-NEXT: s_lshl_b32 s37, s37, 16
-; SI-NEXT: s_lshl_b32 s38, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 23
-; SI-NEXT: s_or_b32 s38, s38, s37
-; SI-NEXT: s_mov_b32 s90, s8
-; SI-NEXT: s_and_b32 s37, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 22
-; SI-NEXT: s_lshl_b32 s37, s37, 16
-; SI-NEXT: s_mov_b32 s28, s8
-; SI-NEXT: s_lshl_b32 s39, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 21
-; SI-NEXT: s_or_b32 s37, s39, s37
-; SI-NEXT: s_and_b32 s39, s8, 0xff
-; SI-NEXT: s_lshl_b32 s39, s39, 16
-; SI-NEXT: s_lshl_b32 s48, s10, 24
-; SI-NEXT: s_or_b32 s39, s48, s39
-; SI-NEXT: s_and_b32 s48, s51, 0xff
-; SI-NEXT: s_lshl_b32 s64, s83, 8
-; SI-NEXT: v_readlane_b32 s8, v46, 8
-; SI-NEXT: s_or_b32 s25, s48, s64
-; SI-NEXT: s_mov_b32 s57, s8
-; SI-NEXT: s_and_b32 s48, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v46, 7
-; SI-NEXT: s_mov_b32 s76, s8
-; SI-NEXT: s_lshl_b32 s64, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v46, 9
-; SI-NEXT: s_lshl_b32 s80, s8, 16
-; SI-NEXT: v_readlane_b32 s8, v46, 10
-; SI-NEXT: s_lshl_b32 s71, s8, 16
-; SI-NEXT: v_readlane_b32 s8, v46, 11
-; SI-NEXT: s_lshl_b32 s48, s48, 16
-; SI-NEXT: s_lshl_b32 s70, s8, 16
-; SI-NEXT: v_readlane_b32 s8, v46, 12
-; SI-NEXT: s_mov_b32 s74, s10
-; SI-NEXT: s_mov_b32 s26, s51
-; SI-NEXT: s_mov_b32 s73, s83
-; SI-NEXT: s_or_b32 s48, s64, s48
-; SI-NEXT: s_lshl_b32 s69, s8, 16
-; SI-NEXT: s_lshl_b32 s68, s6, 16
-; SI-NEXT: s_lshl_b32 s67, s52, 16
-; SI-NEXT: s_lshl_b32 s66, s53, 16
-; SI-NEXT: s_lshl_b32 s65, s54, 16
-; SI-NEXT: s_lshl_b32 s64, s55, 16
-; SI-NEXT: s_lshl_b32 s55, s16, 16
-; SI-NEXT: s_lshl_b32 s54, s19, 16
-; SI-NEXT: s_mov_b32 s19, s5
-; SI-NEXT: s_lshl_b32 s53, s22, 16
-; SI-NEXT: s_lshl_b32 s52, s17, 16
-; SI-NEXT: s_lshl_b32 s51, s20, 16
-; SI-NEXT: s_mov_b32 s20, s4
-; SI-NEXT: s_lshl_b32 s50, s23, 16
-; SI-NEXT: s_lshl_b32 s49, s25, 16
+; SI-NEXT: s_mov_b32 s41, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 24
+; SI-NEXT: s_or_b32 s74, s13, s6
+; SI-NEXT: v_readlane_b32 s6, v46, 22
+; SI-NEXT: s_mov_b32 s79, s80
+; SI-NEXT: s_mov_b32 s59, s77
+; SI-NEXT: s_mov_b32 s21, s18
+; SI-NEXT: s_mov_b32 s18, s60
+; SI-NEXT: s_mov_b32 vcc_lo, s63
+; SI-NEXT: s_lshl_b32 s13, s6, 16
+; SI-NEXT: s_lshl_b32 s42, s93, 16
+; SI-NEXT: s_lshl_b32 s6, s90, 16
+; SI-NEXT: s_lshl_b32 s63, s58, 16
+; SI-NEXT: s_lshl_b32 s73, s45, 16
+; SI-NEXT: s_lshl_b32 s54, s40, 16
+; SI-NEXT: s_lshl_b32 s50, s15, 16
+; SI-NEXT: s_lshl_b32 s38, s51, 16
+; SI-NEXT: s_lshl_b32 s34, s10, 16
+; SI-NEXT: s_lshl_b32 s95, s5, 16
+; SI-NEXT: s_lshl_b32 s90, s29, 16
+; SI-NEXT: s_lshl_b32 s77, s23, 16
+; SI-NEXT: s_lshl_b32 s58, s24, 16
+; SI-NEXT: s_lshl_b32 s45, s25, 16
+; SI-NEXT: s_lshl_b32 s10, s4, 16
+; SI-NEXT: s_lshl_b32 s56, s26, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB89_3
; SI-NEXT: .LBB89_2:
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: v_readlane_b32 s90, v56, 23
+; SI-NEXT: s_mov_b32 s75, s50
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: v_readlane_b32 s28, v56, 22
+; SI-NEXT: s_mov_b32 s79, vcc_lo
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s26, s10
+; SI-NEXT: s_mov_b32 s20, s26
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s74, s5
+; SI-NEXT: s_mov_b32 s91, s93
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s73, s51
+; SI-NEXT: v_readlane_b32 s59, v56, 16
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s57, s4
+; SI-NEXT: v_readlane_b32 s21, v56, 13
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: v_readlane_b32 s72, v56, 15
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 s17, v56, 14
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 s18, v56, 12
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 vcc_lo, v56, 11
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 s41, v56, 10
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s76, s83
; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr13
+; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr69
; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr18
-; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr42
-; SI-NEXT: ; implicit-def: $sgpr19
-; SI-NEXT: ; implicit-def: $sgpr20
-; SI-NEXT: ; implicit-def: $sgpr67
-; SI-NEXT: ; implicit-def: $sgpr43
-; SI-NEXT: ; implicit-def: $sgpr45
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr66
-; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr41
-; SI-NEXT: ; implicit-def: $sgpr59
-; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr27
-; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr75
-; SI-NEXT: ; implicit-def: $sgpr64
-; SI-NEXT: ; implicit-def: $sgpr29
-; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr24
-; SI-NEXT: ; implicit-def: $sgpr55
-; SI-NEXT: ; implicit-def: $sgpr21
-; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: ; implicit-def: $sgpr63
+; SI-NEXT: ; implicit-def: $sgpr73
; SI-NEXT: ; implicit-def: $sgpr54
-; SI-NEXT: ; implicit-def: $sgpr92
-; SI-NEXT: ; implicit-def: $sgpr91
-; SI-NEXT: ; implicit-def: $sgpr93
-; SI-NEXT: ; implicit-def: $sgpr53
-; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $sgpr94
-; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr52
-; SI-NEXT: ; implicit-def: $sgpr30
-; SI-NEXT: ; implicit-def: $vcc_hi
-; SI-NEXT: ; implicit-def: $sgpr31
-; SI-NEXT: ; implicit-def: $sgpr51
-; SI-NEXT: ; implicit-def: $sgpr35
-; SI-NEXT: ; implicit-def: $sgpr34
-; SI-NEXT: ; implicit-def: $sgpr36
; SI-NEXT: ; implicit-def: $sgpr50
-; SI-NEXT: ; implicit-def: $sgpr38
-; SI-NEXT: ; implicit-def: $sgpr37
-; SI-NEXT: ; implicit-def: $sgpr39
; SI-NEXT: ; implicit-def: $sgpr49
-; SI-NEXT: ; implicit-def: $sgpr48
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr16
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr11
+; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr22
+; SI-NEXT: ; implicit-def: $sgpr95
+; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr94
+; SI-NEXT: ; implicit-def: $sgpr90
+; SI-NEXT: ; implicit-def: $sgpr92
+; SI-NEXT: ; implicit-def: $sgpr88
+; SI-NEXT: ; implicit-def: $sgpr89
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: ; implicit-def: $sgpr58
+; SI-NEXT: ; implicit-def: $sgpr61
+; SI-NEXT: ; implicit-def: $sgpr47
+; SI-NEXT: ; implicit-def: $sgpr57
+; SI-NEXT: ; implicit-def: $sgpr45
+; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr44
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr9
+; SI-NEXT: ; implicit-def: $sgpr56
+; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: .LBB89_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: v_readlane_b32 s81, v47, 59
-; SI-NEXT: v_readlane_b32 s82, v47, 60
-; SI-NEXT: v_readlane_b32 s84, v47, 61
-; SI-NEXT: v_readlane_b32 s83, v47, 62
-; SI-NEXT: v_readlane_b32 s85, v47, 63
-; SI-NEXT: v_readlane_b32 s87, v46, 0
-; SI-NEXT: v_readlane_b32 s86, v46, 1
-; SI-NEXT: v_readlane_b32 s96, v46, 2
-; SI-NEXT: v_readlane_b32 s98, v46, 3
-; SI-NEXT: v_readlane_b32 s97, v46, 4
-; SI-NEXT: v_readlane_b32 s99, v46, 5
-; SI-NEXT: v_readlane_b32 s6, v46, 6
+; SI-NEXT: v_readlane_b32 s4, v46, 1
+; SI-NEXT: v_readlane_b32 s15, v46, 7
+; SI-NEXT: s_mov_b32 s37, s7
+; SI-NEXT: s_mov_b32 s7, s73
+; SI-NEXT: v_readlane_b32 s84, v47, 60
+; SI-NEXT: v_readlane_b32 s85, v47, 61
+; SI-NEXT: v_readlane_b32 s81, v47, 62
+; SI-NEXT: v_readlane_b32 s83, v47, 63
+; SI-NEXT: v_readlane_b32 s82, v46, 0
+; SI-NEXT: v_readlane_b32 s43, v46, 2
+; SI-NEXT: v_readlane_b32 s80, v46, 3
+; SI-NEXT: v_readlane_b32 s69, v46, 4
+; SI-NEXT: v_readlane_b32 s71, v46, 5
+; SI-NEXT: v_readlane_b32 s70, v46, 6
+; SI-NEXT: v_readlane_b32 s52, v46, 8
+; SI-NEXT: v_readlane_b32 s53, v46, 9
+; SI-NEXT: v_readlane_b32 s48, v46, 10
+; SI-NEXT: v_readlane_b32 s51, v46, 11
+; SI-NEXT: v_readlane_b32 s39, v46, 12
+; SI-NEXT: s_mov_b32 s30, s55
+; SI-NEXT: s_mov_b32 s35, s11
+; SI-NEXT: s_mov_b32 s93, s65
+; SI-NEXT: s_mov_b32 vcc_hi, s64
+; SI-NEXT: s_mov_b32 s5, s74
+; SI-NEXT: s_mov_b32 s74, s4
+; SI-NEXT: s_mov_b32 s4, s56
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: v_readlane_b32 s68, v46, 13
+; SI-NEXT: v_readlane_b32 s67, v46, 14
+; SI-NEXT: v_readlane_b32 s66, v46, 15
+; SI-NEXT: s_mov_b32 s40, s42
+; SI-NEXT: s_mov_b32 s42, s7
+; SI-NEXT: v_readlane_b32 s65, v46, 16
+; SI-NEXT: v_readlane_b32 s64, v46, 17
+; SI-NEXT: v_readlane_b32 s55, v46, 18
+; SI-NEXT: s_mov_b32 s56, s13
+; SI-NEXT: v_readlane_b32 s60, v47, 50
+; SI-NEXT: v_readlane_b32 s96, v47, 51
+; SI-NEXT: v_readlane_b32 s97, v47, 52
+; SI-NEXT: v_readlane_b32 s98, v47, 53
+; SI-NEXT: v_readlane_b32 s29, v47, 54
+; SI-NEXT: v_readlane_b32 s99, v47, 55
+; SI-NEXT: v_readlane_b32 s28, v47, 58
; SI-NEXT: s_cbranch_scc1 .LBB89_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s4, s26, 3
+; SI-NEXT: s_add_i32 s4, s21, 3
; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s5, s73, 8
-; SI-NEXT: s_add_i32 s6, s57, 3
+; SI-NEXT: s_lshl_b32 s5, s18, 8
+; SI-NEXT: s_add_i32 s6, vcc_lo, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s5, s76, 24
+; SI-NEXT: s_lshl_b32 s5, s41, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s5, s5, s6
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s5, s90, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 21
-; SI-NEXT: s_and_b32 s5, s5, 0xff
-; SI-NEXT: s_lshl_b32 s6, s28, 8
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_or_b32 s5, s5, s4
+; SI-NEXT: s_add_i32 s4, s20, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s91, 8
+; SI-NEXT: s_add_i32 s7, s72, 3
+; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_and_b32 s7, s7, 0xff
-; SI-NEXT: s_lshl_b32 s6, s74, 24
+; SI-NEXT: s_lshl_b32 s6, s17, 24
; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_addk_i32 s5, 0x300
+; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s6, s6, s7
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v56, 27
-; SI-NEXT: s_add_i32 s21, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 26
-; SI-NEXT: v_readlane_b32 s8, v56, 25
-; SI-NEXT: s_and_b32 s6, s21, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s8, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 24
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 31
-; SI-NEXT: s_add_i32 s10, s7, 3
-; SI-NEXT: v_readlane_b32 s8, v56, 30
-; SI-NEXT: s_and_b32 s7, s10, 0xff
-; SI-NEXT: s_lshl_b32 s10, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v56, 28
-; SI-NEXT: s_or_b32 s7, s10, s7
-; SI-NEXT: s_lshl_b32 s10, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 29
-; SI-NEXT: s_add_i32 s12, s8, 3
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_or_b32 s6, s6, s4
+; SI-NEXT: s_add_i32 s4, s75, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s7, s79, 8
+; SI-NEXT: s_or_b32 s7, s7, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s8, s59, 24
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s7, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 50
-; SI-NEXT: s_or_b32 s10, s10, s12
+; SI-NEXT: s_or_b32 s4, s8, s4
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_add_i32 s57, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 41
-; SI-NEXT: s_or_b32 s7, s10, s7
-; SI-NEXT: s_and_b32 s10, s57, 0xff
-; SI-NEXT: s_lshl_b32 s12, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 33
-; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_lshl_b32 s12, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 35
-; SI-NEXT: s_add_i32 s26, s8, 3
-; SI-NEXT: s_and_b32 s13, s26, 0xff
-; SI-NEXT: v_readlane_b32 s8, v47, 55
-; SI-NEXT: s_lshl_b32 s13, s13, 16
-; SI-NEXT: s_add_i32 s73, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 51
-; SI-NEXT: s_or_b32 s12, s12, s13
+; SI-NEXT: s_or_b32 s7, s4, s7
+; SI-NEXT: v_readlane_b32 s4, v56, 19
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s8, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 48
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s8, s4, s8
+; SI-NEXT: v_readlane_b32 s4, v56, 18
+; SI-NEXT: s_lshl_b32 s9, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 47
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s8, 0x300
+; SI-NEXT: s_or_b32 s4, s9, s4
+; SI-NEXT: s_and_b32 s8, s8, 0xffff
+; SI-NEXT: s_or_b32 s8, s4, s8
+; SI-NEXT: v_readlane_b32 s4, v56, 23
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s9, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v56, 22
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s9, s4, s9
+; SI-NEXT: v_readlane_b32 s4, v56, 20
+; SI-NEXT: s_lshl_b32 s10, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v56, 21
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s9, 0x300
+; SI-NEXT: s_or_b32 s4, s10, s4
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: s_or_b32 s9, s4, s9
+; SI-NEXT: v_readlane_b32 s4, v56, 27
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s10, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v56, 26
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s10, s4, s10
+; SI-NEXT: v_readlane_b32 s4, v56, 24
+; SI-NEXT: s_lshl_b32 s11, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v56, 25
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s10, 0x300
-; SI-NEXT: s_lshl_b32 s13, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 39
+; SI-NEXT: s_or_b32 s4, s11, s4
; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 40
-; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_and_b32 s12, s73, 0xff
-; SI-NEXT: s_add_i32 s11, s8, 3
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s11, s11, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_addk_i32 s12, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 56
-; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: s_and_b32 s11, s12, 0xffff
-; SI-NEXT: s_add_i32 s74, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 53
-; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: s_and_b32 s11, s74, 0xff
-; SI-NEXT: s_lshl_b32 s12, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 43
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_lshl_b32 s12, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 46
-; SI-NEXT: s_add_i32 s43, s8, 3
-; SI-NEXT: s_and_b32 s13, s43, 0xff
-; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s10, s4, s10
+; SI-NEXT: v_readlane_b32 s4, v56, 31
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s11, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v56, 30
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s11, s4, s11
+; SI-NEXT: v_readlane_b32 s4, v56, 28
+; SI-NEXT: s_lshl_b32 s12, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v56, 29
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s11, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 58
-; SI-NEXT: s_or_b32 s12, s12, s13
+; SI-NEXT: s_or_b32 s4, s12, s4
; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_add_i32 s78, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 57
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_and_b32 s12, s78, 0xff
-; SI-NEXT: s_lshl_b32 s13, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 44
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_lshl_b32 s13, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 47
-; SI-NEXT: s_add_i32 s45, s8, 3
-; SI-NEXT: s_and_b32 s15, s45, 0xff
-; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s11, s4, s11
+; SI-NEXT: v_readlane_b32 s4, v47, 46
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s12, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 45
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s12, s4, s12
+; SI-NEXT: v_readlane_b32 s4, v47, 44
+; SI-NEXT: s_lshl_b32 s13, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 43
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s12, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 54
-; SI-NEXT: s_or_b32 s13, s13, s15
+; SI-NEXT: s_or_b32 s4, s13, s4
; SI-NEXT: s_and_b32 s12, s12, 0xffff
-; SI-NEXT: s_add_i32 s72, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 52
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s13, s72, 0xff
-; SI-NEXT: s_lshl_b32 s15, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 49
-; SI-NEXT: s_or_b32 s13, s15, s13
-; SI-NEXT: s_lshl_b32 s15, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 48
-; SI-NEXT: s_add_i32 s47, s8, 3
-; SI-NEXT: s_and_b32 s18, s47, 0xff
-; SI-NEXT: s_lshl_b32 s18, s18, 16
+; SI-NEXT: s_or_b32 s12, s4, s12
+; SI-NEXT: v_readlane_b32 s4, v47, 41
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s13, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 40
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s13, s4, s13
+; SI-NEXT: v_readlane_b32 s4, v47, 42
+; SI-NEXT: s_lshl_b32 s14, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 39
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s13, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 45
-; SI-NEXT: s_or_b32 s15, s15, s18
+; SI-NEXT: s_or_b32 s4, s14, s4
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_add_i32 s42, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 42
-; SI-NEXT: s_or_b32 s13, s15, s13
-; SI-NEXT: s_and_b32 s15, s42, 0xff
-; SI-NEXT: s_lshl_b32 s14, s8, 8
-; SI-NEXT: s_or_b32 s14, s14, s15
-; SI-NEXT: v_readlane_b32 s15, v47, 37
-; SI-NEXT: s_add_i32 s29, s15, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 38
-; SI-NEXT: s_and_b32 s15, s29, 0xff
-; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s13, s4, s13
+; SI-NEXT: v_readlane_b32 s4, v47, 37
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s14, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 36
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s14, s4, s14
+; SI-NEXT: v_readlane_b32 s4, v47, 38
+; SI-NEXT: s_lshl_b32 s15, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 35
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s14, 0x300
-; SI-NEXT: s_or_b32 s8, s8, s15
+; SI-NEXT: s_or_b32 s4, s15, s4
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_or_b32 s14, s8, s14
-; SI-NEXT: v_readlane_b32 s8, v47, 36
-; SI-NEXT: s_add_i32 s24, s8, 3
-; SI-NEXT: v_readlane_b32 s15, v47, 34
-; SI-NEXT: v_readlane_b32 s16, v47, 31
-; SI-NEXT: s_and_b32 s8, s24, 0xff
-; SI-NEXT: s_lshl_b32 s15, s15, 8
-; SI-NEXT: s_add_i32 s17, s16, 3
-; SI-NEXT: s_or_b32 s8, s15, s8
-; SI-NEXT: v_readlane_b32 s15, v47, 32
-; SI-NEXT: s_and_b32 s16, s17, 0xff
-; SI-NEXT: s_lshl_b32 s15, s15, 24
+; SI-NEXT: s_or_b32 s14, s4, s14
+; SI-NEXT: v_readlane_b32 s4, v47, 34
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s15, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 32
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s15, s4, s15
+; SI-NEXT: v_readlane_b32 s4, v47, 33
+; SI-NEXT: s_lshl_b32 s16, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 31
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s15, 0x300
+; SI-NEXT: s_or_b32 s4, s16, s4
+; SI-NEXT: s_and_b32 s15, s15, 0xffff
+; SI-NEXT: s_or_b32 s15, s4, s15
+; SI-NEXT: v_readlane_b32 s4, v47, 28
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s16, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 30
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s16, s4, s16
+; SI-NEXT: v_readlane_b32 s4, v47, 29
+; SI-NEXT: s_lshl_b32 s17, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 27
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_or_b32 s4, s17, s4
+; SI-NEXT: s_and_b32 s16, s16, 0xffff
+; SI-NEXT: s_or_b32 s16, s4, s16
+; SI-NEXT: s_add_i32 s4, s5, 0x3000000
+; SI-NEXT: s_add_i32 s5, s6, 0x3000000
+; SI-NEXT: s_add_i32 s6, s7, 0x3000000
+; SI-NEXT: s_add_i32 s7, s8, 0x3000000
+; SI-NEXT: s_add_i32 s8, s9, 0x3000000
+; SI-NEXT: s_add_i32 s9, s10, 0x3000000
+; SI-NEXT: s_add_i32 s10, s11, 0x3000000
+; SI-NEXT: s_add_i32 s11, s12, 0x3000000
+; SI-NEXT: s_add_i32 s12, s13, 0x3000000
+; SI-NEXT: s_add_i32 s13, s14, 0x3000000
+; SI-NEXT: s_add_i32 s14, s15, 0x3000000
+; SI-NEXT: s_add_i32 s15, s16, 0x3000000
+; SI-NEXT: v_readlane_b32 s16, v47, 19
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 18
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 17
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 16
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_or_b32 s15, s15, s16
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_or_b32 s15, s15, s8
-; SI-NEXT: v_readlane_b32 s8, v47, 30
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: v_readlane_b32 s16, v47, 29
-; SI-NEXT: v_readlane_b32 s17, v47, 27
-; SI-NEXT: s_and_b32 s8, s8, 0xff
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s40, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 15
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 11
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 7
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 6
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s41, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 5
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 4
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 3
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 2
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s44, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 0
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 1
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v56, 63
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v56, 62
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s45, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v56, 61
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v56, 59
; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v56, 60
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v56, 58
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s46, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v56, 56
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v56, 57
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 26
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 25
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s47, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 24
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 23
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 22
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 21
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: v_readlane_b32 s17, v47, 20
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s18, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v47, 14
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s18, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v47, 13
+; SI-NEXT: s_lshl_b32 s20, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v47, 12
; SI-NEXT: s_add_i32 s17, s17, 3
-; SI-NEXT: s_or_b32 s8, s16, s8
-; SI-NEXT: v_readlane_b32 s16, v47, 28
; SI-NEXT: s_and_b32 s17, s17, 0xff
-; SI-NEXT: s_lshl_b32 s16, s16, 24
; SI-NEXT: s_lshl_b32 s17, s17, 16
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_add_i32 s23, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v47, 26
-; SI-NEXT: s_or_b32 s16, s16, s8
-; SI-NEXT: s_add_i32 s22, s7, 0x3000000
-; SI-NEXT: s_add_i32 s8, s10, 0x3000000
-; SI-NEXT: s_add_i32 s10, s11, 0x3000000
-; SI-NEXT: s_add_i32 s11, s12, 0x3000000
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 25
-; SI-NEXT: v_readlane_b32 s12, v47, 23
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 24
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s17, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 22
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 21
-; SI-NEXT: v_readlane_b32 s12, v47, 19
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 20
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s21, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 18
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 17
-; SI-NEXT: v_readlane_b32 s12, v47, 15
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 16
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s24, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 14
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 13
-; SI-NEXT: v_readlane_b32 s12, v47, 11
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 12
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s25, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 10
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 9
-; SI-NEXT: v_readlane_b32 s12, v47, 7
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 8
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s26, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 6
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_and_b32 s7, s6, 0xff
-; SI-NEXT: v_readlane_b32 s6, v56, 32
-; SI-NEXT: v_readlane_b32 s12, v47, 4
-; SI-NEXT: s_lshl_b32 s6, s6, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s6, s7
-; SI-NEXT: v_readlane_b32 s7, v47, 5
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s29, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 3
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 2
-; SI-NEXT: v_readlane_b32 s12, v47, 0
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 1
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s40, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v56, 63
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 62
-; SI-NEXT: v_readlane_b32 s12, v56, 60
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 61
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s27, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 9
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 59
-; SI-NEXT: v_readlane_b32 s12, v56, 57
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 58
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s12, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 13
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 12
-; SI-NEXT: v_readlane_b32 s18, v56, 11
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 10
-; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s18
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s28, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 17
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 16
-; SI-NEXT: v_readlane_b32 s18, v56, 15
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 14
-; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s18
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s20, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 47
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 20
-; SI-NEXT: v_readlane_b32 s18, v56, 19
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 18
-; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s18
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s7, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 49
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s18, v56, 45
-; SI-NEXT: v_readlane_b32 s19, v56, 39
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s18, s18, 8
-; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_or_b32 s6, s18, s6
-; SI-NEXT: v_readlane_b32 s18, v56, 38
-; SI-NEXT: s_and_b32 s19, s19, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s18, s18, 24
-; SI-NEXT: s_lshl_b32 s19, s19, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s18, s18, s19
-; SI-NEXT: s_or_b32 s6, s18, s6
-; SI-NEXT: s_add_i32 s18, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 51
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s19, v56, 48
-; SI-NEXT: v_readlane_b32 s41, v56, 41
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s19, s19, 8
-; SI-NEXT: s_add_i32 s41, s41, 3
-; SI-NEXT: s_or_b32 s6, s19, s6
-; SI-NEXT: v_readlane_b32 s19, v56, 40
-; SI-NEXT: s_and_b32 s41, s41, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s19, s19, 24
-; SI-NEXT: s_lshl_b32 s41, s41, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s19, s19, s41
-; SI-NEXT: s_or_b32 s6, s19, s6
-; SI-NEXT: v_readlane_b32 s19, v56, 52
-; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: v_readlane_b32 s41, v56, 50
-; SI-NEXT: v_readlane_b32 s42, v56, 43
-; SI-NEXT: s_and_b32 s19, s19, 0xff
-; SI-NEXT: s_lshl_b32 s41, s41, 8
-; SI-NEXT: s_add_i32 s42, s42, 3
-; SI-NEXT: s_or_b32 s19, s41, s19
-; SI-NEXT: v_readlane_b32 s41, v56, 42
+; SI-NEXT: s_or_b32 s17, s20, s17
+; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_add_i32 s18, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 55
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s20, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v47, 10
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s20
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s21, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v47, 9
+; SI-NEXT: s_lshl_b32 s20, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v47, 8
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s20, s17
+; SI-NEXT: s_or_b32 s17, s17, s21
+; SI-NEXT: s_add_i32 s57, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s20, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 52
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s20
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s20, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 53
+; SI-NEXT: s_lshl_b32 s21, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 51
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s21, s17
+; SI-NEXT: s_or_b32 s17, s17, s20
+; SI-NEXT: s_add_i32 s20, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 50
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s21, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 48
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s21
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s21, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 49
+; SI-NEXT: s_lshl_b32 s22, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 47
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s22, s17
+; SI-NEXT: s_or_b32 s17, s17, s21
+; SI-NEXT: s_add_i32 s21, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 46
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s22, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 45
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s22
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s22, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 44
+; SI-NEXT: s_lshl_b32 s23, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 43
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s23, s17
+; SI-NEXT: s_or_b32 s17, s17, s22
+; SI-NEXT: s_add_i32 s22, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 42
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s23, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s23
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s23, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 41
+; SI-NEXT: s_lshl_b32 s24, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 38
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s24, s17
+; SI-NEXT: s_or_b32 s17, s17, s23
+; SI-NEXT: s_add_i32 s23, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 37
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s24, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 35
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s24
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s24, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 36
+; SI-NEXT: s_lshl_b32 s25, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 34
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s25, s17
+; SI-NEXT: s_or_b32 s17, s17, s24
+; SI-NEXT: s_add_i32 s24, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 4
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s25, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 3
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s25
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s25, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 32
+; SI-NEXT: s_lshl_b32 s26, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 33
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s26, s17
+; SI-NEXT: s_or_b32 s17, s17, s25
+; SI-NEXT: s_add_i32 s25, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 8
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s26, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 7
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s26
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s26, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 5
+; SI-NEXT: s_lshl_b32 s27, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 6
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s27, s17
+; SI-NEXT: s_or_b32 s17, s17, s26
+; SI-NEXT: s_add_i32 s26, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v47, 59
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s27, s28, 8
+; SI-NEXT: s_or_b32 s17, s27, s17
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s27, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 9
+; SI-NEXT: s_lshl_b32 s28, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 39
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s28, s17
+; SI-NEXT: s_or_b32 s17, s17, s27
+; SI-NEXT: v_readlane_b32 s27, v47, 57
+; SI-NEXT: s_add_i32 s27, s27, 3
+; SI-NEXT: v_readlane_b32 s28, v47, 56
+; SI-NEXT: s_and_b32 s27, s27, 0xff
+; SI-NEXT: s_lshl_b32 s28, s28, 8
+; SI-NEXT: s_add_i32 s29, s29, 3
+; SI-NEXT: s_or_b32 s27, s28, s27
+; SI-NEXT: s_and_b32 s29, s29, 0xff
+; SI-NEXT: s_addk_i32 s27, 0x300
+; SI-NEXT: s_lshl_b32 s28, s99, 24
+; SI-NEXT: s_lshl_b32 s29, s29, 16
+; SI-NEXT: s_and_b32 s27, s27, 0xffff
+; SI-NEXT: s_or_b32 s28, s28, s29
+; SI-NEXT: s_or_b32 s27, s28, s27
+; SI-NEXT: s_add_i32 s28, s98, 3
+; SI-NEXT: s_and_b32 s28, s28, 0xff
+; SI-NEXT: s_lshl_b32 s29, s97, 8
+; SI-NEXT: s_add_i32 s42, s60, 3
+; SI-NEXT: s_or_b32 s28, s29, s28
; SI-NEXT: s_and_b32 s42, s42, 0xff
-; SI-NEXT: s_addk_i32 s19, 0x300
-; SI-NEXT: s_lshl_b32 s41, s41, 24
+; SI-NEXT: s_addk_i32 s28, 0x300
+; SI-NEXT: s_lshl_b32 s29, s96, 24
; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_and_b32 s19, s19, 0xffff
-; SI-NEXT: s_or_b32 s41, s41, s42
-; SI-NEXT: s_or_b32 s19, s41, s19
-; SI-NEXT: v_readlane_b32 s41, v56, 55
-; SI-NEXT: s_add_i32 s41, s41, 3
-; SI-NEXT: v_readlane_b32 s42, v56, 53
-; SI-NEXT: v_readlane_b32 s43, v56, 46
-; SI-NEXT: s_and_b32 s41, s41, 0xff
+; SI-NEXT: s_and_b32 s28, s28, 0xffff
+; SI-NEXT: s_or_b32 s29, s29, s42
+; SI-NEXT: s_or_b32 s28, s29, s28
+; SI-NEXT: v_readlane_b32 s29, v56, 2
+; SI-NEXT: s_add_i32 s29, s29, 3
+; SI-NEXT: v_readlane_b32 s42, v56, 1
+; SI-NEXT: v_readlane_b32 s43, v47, 49
+; SI-NEXT: s_and_b32 s29, s29, 0xff
; SI-NEXT: s_lshl_b32 s42, s42, 8
; SI-NEXT: s_add_i32 s43, s43, 3
-; SI-NEXT: s_or_b32 s41, s42, s41
-; SI-NEXT: v_readlane_b32 s42, v56, 44
+; SI-NEXT: s_or_b32 s29, s42, s29
+; SI-NEXT: v_readlane_b32 s19, v56, 0
; SI-NEXT: s_and_b32 s43, s43, 0xff
-; SI-NEXT: s_addk_i32 s41, 0x300
-; SI-NEXT: s_lshl_b32 s42, s42, 24
+; SI-NEXT: s_addk_i32 s29, 0x300
+; SI-NEXT: s_lshl_b32 s42, s19, 24
; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_and_b32 s41, s41, 0xffff
+; SI-NEXT: s_and_b32 s29, s29, 0xffff
; SI-NEXT: s_or_b32 s42, s42, s43
-; SI-NEXT: s_or_b32 s41, s42, s41
-; SI-NEXT: v_readlane_b32 s42, v56, 1
-; SI-NEXT: s_add_i32 s42, s42, 3
-; SI-NEXT: v_readlane_b32 s43, v56, 0
-; SI-NEXT: v_readlane_b32 s44, v56, 56
-; SI-NEXT: s_and_b32 s42, s42, 0xff
-; SI-NEXT: s_lshl_b32 s43, s43, 8
-; SI-NEXT: s_add_i32 s44, s44, 3
-; SI-NEXT: s_or_b32 s42, s43, s42
-; SI-NEXT: v_readlane_b32 s43, v56, 54
-; SI-NEXT: s_and_b32 s44, s44, 0xff
-; SI-NEXT: s_addk_i32 s42, 0x300
-; SI-NEXT: s_lshl_b32 s43, s43, 24
-; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_and_b32 s42, s42, 0xffff
-; SI-NEXT: s_or_b32 s43, s43, s44
-; SI-NEXT: s_or_b32 s42, s43, s42
-; SI-NEXT: v_readlane_b32 s43, v56, 5
-; SI-NEXT: s_add_i32 s43, s43, 3
-; SI-NEXT: v_readlane_b32 s44, v56, 4
-; SI-NEXT: v_readlane_b32 s45, v56, 3
-; SI-NEXT: s_and_b32 s43, s43, 0xff
-; SI-NEXT: s_lshl_b32 s44, s44, 8
-; SI-NEXT: s_add_i32 s45, s45, 3
-; SI-NEXT: s_or_b32 s43, s44, s43
-; SI-NEXT: v_readlane_b32 s44, v56, 2
-; SI-NEXT: s_and_b32 s45, s45, 0xff
-; SI-NEXT: s_addk_i32 s43, 0x300
-; SI-NEXT: s_lshl_b32 s44, s44, 24
-; SI-NEXT: s_lshl_b32 s45, s45, 16
-; SI-NEXT: s_and_b32 s43, s43, 0xffff
-; SI-NEXT: s_or_b32 s44, s44, s45
-; SI-NEXT: s_or_b32 s43, s44, s43
-; SI-NEXT: v_readlane_b32 s44, v56, 37
-; SI-NEXT: s_add_i32 s44, s44, 3
-; SI-NEXT: v_readlane_b32 s45, v56, 8
-; SI-NEXT: v_readlane_b32 s46, v56, 7
-; SI-NEXT: s_and_b32 s44, s44, 0xff
-; SI-NEXT: s_lshl_b32 s45, s45, 8
-; SI-NEXT: s_add_i32 s46, s46, 3
-; SI-NEXT: s_or_b32 s44, s45, s44
-; SI-NEXT: v_readlane_b32 s45, v56, 6
-; SI-NEXT: s_and_b32 s46, s46, 0xff
-; SI-NEXT: s_addk_i32 s44, 0x300
-; SI-NEXT: s_lshl_b32 s45, s45, 24
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_and_b32 s44, s44, 0xffff
-; SI-NEXT: s_or_b32 s45, s45, s46
-; SI-NEXT: s_or_b32 s44, s45, s44
-; SI-NEXT: v_readlane_b32 s45, v56, 36
-; SI-NEXT: s_add_i32 s45, s45, 3
-; SI-NEXT: v_readlane_b32 s46, v56, 35
-; SI-NEXT: v_readlane_b32 s47, v56, 33
-; SI-NEXT: s_and_b32 s45, s45, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 8
-; SI-NEXT: s_add_i32 s47, s47, 3
-; SI-NEXT: s_or_b32 s45, s46, s45
-; SI-NEXT: v_readlane_b32 s46, v56, 34
-; SI-NEXT: s_and_b32 s47, s47, 0xff
-; SI-NEXT: s_addk_i32 s45, 0x300
-; SI-NEXT: s_lshl_b32 s46, s46, 24
-; SI-NEXT: s_lshl_b32 s47, s47, 16
-; SI-NEXT: s_and_b32 s45, s45, 0xffff
-; SI-NEXT: s_or_b32 s46, s46, s47
-; SI-NEXT: s_or_b32 s45, s46, s45
-; SI-NEXT: s_add_i32 s4, s4, 0x3000000
-; SI-NEXT: s_add_i32 s5, s5, 0x3000000
-; SI-NEXT: s_add_i32 s9, s9, 0x3000000
-; SI-NEXT: s_add_i32 s13, s13, 0x3000000
-; SI-NEXT: s_add_i32 s14, s14, 0x3000000
-; SI-NEXT: s_add_i32 s15, s15, 0x3000000
-; SI-NEXT: s_add_i32 s16, s16, 0x3000000
; SI-NEXT: s_add_i32 s17, s17, 0x3000000
-; SI-NEXT: s_add_i32 s21, s21, 0x3000000
-; SI-NEXT: s_add_i32 s24, s24, 0x3000000
-; SI-NEXT: s_add_i32 s25, s25, 0x3000000
-; SI-NEXT: s_add_i32 s26, s26, 0x3000000
-; SI-NEXT: s_add_i32 s29, s29, 0x3000000
+; SI-NEXT: s_or_b32 s29, s42, s29
; SI-NEXT: s_add_i32 s40, s40, 0x3000000
-; SI-NEXT: s_add_i32 s6, s6, 0x3000000
-; SI-NEXT: s_add_i32 s19, s19, 0x3000000
; SI-NEXT: s_add_i32 s41, s41, 0x3000000
-; SI-NEXT: s_add_i32 s42, s42, 0x3000000
-; SI-NEXT: s_add_i32 s43, s43, 0x3000000
; SI-NEXT: s_add_i32 s44, s44, 0x3000000
; SI-NEXT: s_add_i32 s45, s45, 0x3000000
-; SI-NEXT: s_and_b32 s82, s45, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s45, 16
-; SI-NEXT: s_and_b32 s83, s44, 0xffff0000
-; SI-NEXT: s_lshl_b32 s80, s44, 16
-; SI-NEXT: s_and_b32 s85, s43, 0xffff0000
-; SI-NEXT: s_lshl_b32 s84, s43, 16
-; SI-NEXT: s_and_b32 s86, s42, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s42, 16
-; SI-NEXT: s_and_b32 s96, s41, 0xffff0000
-; SI-NEXT: s_lshl_b32 s87, s41, 16
-; SI-NEXT: s_and_b32 s97, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s70, s19, 16
-; SI-NEXT: s_and_b32 s99, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s98, s6, 16
-; SI-NEXT: s_and_b32 s6, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s18, 16
-; SI-NEXT: s_and_b32 s18, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s42, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s68, s20, 16
-; SI-NEXT: s_and_b32 s20, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s19, s28, 16
-; SI-NEXT: s_and_b32 s43, s12, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s12, 16
-; SI-NEXT: s_and_b32 s12, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s27, 16
-; SI-NEXT: s_and_b32 s44, s40, 0xffff0000
-; SI-NEXT: s_lshl_b32 s66, s40, 16
-; SI-NEXT: s_and_b32 s59, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s29, 16
-; SI-NEXT: s_and_b32 s27, s26, 0xffff0000
-; SI-NEXT: s_lshl_b32 s65, s26, 16
-; SI-NEXT: s_and_b32 s75, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 s62, s25, 16
-; SI-NEXT: s_and_b32 s29, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s64, s24, 16
-; SI-NEXT: s_and_b32 s24, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s46, s21, 16
-; SI-NEXT: s_and_b32 s21, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s55, s17, 16
-; SI-NEXT: s_and_b32 s40, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s88, s16, 16
-; SI-NEXT: s_and_b32 s92, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s54, s15, 16
-; SI-NEXT: s_and_b32 s93, s14, 0xffff0000
-; SI-NEXT: s_lshl_b32 s91, s14, 16
-; SI-NEXT: s_and_b32 s95, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s53, s13, 16
-; SI-NEXT: s_and_b32 vcc_lo, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s11, 16
-; SI-NEXT: s_and_b32 s30, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 s52, s10, 16
-; SI-NEXT: s_and_b32 s31, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_hi, s9, 16
-; SI-NEXT: s_and_b32 s35, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s51, s8, 16
-; SI-NEXT: s_and_b32 s36, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s34, s22, 16
-; SI-NEXT: s_and_b32 s38, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s50, s23, 16
-; SI-NEXT: s_and_b32 s39, s5, 0xffff0000
-; SI-NEXT: s_lshl_b32 s37, s5, 16
-; SI-NEXT: s_and_b32 s48, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s49, s4, 16
+; SI-NEXT: s_add_i32 s46, s46, 0x3000000
+; SI-NEXT: s_add_i32 s47, s47, 0x3000000
+; SI-NEXT: s_add_i32 s16, s16, 0x3000000
+; SI-NEXT: s_add_i32 s27, s27, 0x3000000
+; SI-NEXT: s_add_i32 s28, s28, 0x3000000
+; SI-NEXT: s_add_i32 s29, s29, 0x3000000
+; SI-NEXT: s_and_b32 s43, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_and_b32 s71, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s25, s25, 16
+; SI-NEXT: s_and_b32 s85, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s84, s29, 16
+; SI-NEXT: s_and_b32 s83, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s56, s28, 16
+; SI-NEXT: s_and_b32 s82, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s81, s27, 16
+; SI-NEXT: s_and_b32 s80, s26, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s26, 16
+; SI-NEXT: s_and_b32 s70, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 s69, s24, 16
+; SI-NEXT: s_and_b32 s73, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s23, 16
+; SI-NEXT: s_and_b32 s68, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s67, s22, 16
+; SI-NEXT: s_and_b32 s66, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s21, 16
+; SI-NEXT: s_and_b32 s65, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s20, 16
+; SI-NEXT: s_and_b32 s55, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s57, 16
+; SI-NEXT: s_and_b32 s53, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s18, 16
+; SI-NEXT: s_and_b32 s51, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s16, 16
+; SI-NEXT: s_and_b32 s49, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s47, 16
+; SI-NEXT: s_and_b32 s39, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s46, 16
+; SI-NEXT: s_and_b32 s37, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s16, s45, 16
+; SI-NEXT: s_and_b32 s35, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s44, 16
+; SI-NEXT: s_and_b32 s22, s41, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s41, 16
+; SI-NEXT: s_and_b32 vcc_hi, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s40, 16
+; SI-NEXT: s_mov_b32 s40, s17
+; SI-NEXT: s_and_b32 s94, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s15, 16
+; SI-NEXT: s_and_b32 s92, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s90, s14, 16
+; SI-NEXT: s_and_b32 s89, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s88, s13, 16
+; SI-NEXT: s_and_b32 s78, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s12, 16
+; SI-NEXT: s_and_b32 s76, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s62, s11, 16
+; SI-NEXT: s_and_b32 s61, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s58, s10, 16
+; SI-NEXT: s_and_b32 s57, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s9, 16
+; SI-NEXT: s_and_b32 s46, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s45, s8, 16
+; SI-NEXT: s_and_b32 s44, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s7, 16
+; SI-NEXT: s_and_b32 s12, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s6, 16
+; SI-NEXT: s_mov_b32 s6, s25
+; SI-NEXT: s_and_b32 s9, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s8, s5, 16
+; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: .LBB89_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s85
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s84
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
; SI-NEXT: v_mul_f32_e64 v1, 1.0, s83
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s56
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s82
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s81
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s40
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s96
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s80
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:340 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:344 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:348 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:352 ; 4-byte Folded Reload
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s74
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s97
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s71
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s6
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s99
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s70
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s98
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s69
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s73
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s63
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s18
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s68
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s67
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s66
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s42
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s20
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s19
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s64
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s55
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s54
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s52
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s50
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s49
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s48
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s27
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s38
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s16
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s29
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s64
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s34
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s24
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s22
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s30
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s21
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, vcc_hi
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s55
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s95
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s93
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
; SI-NEXT: v_mul_f32_e64 v21, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s54
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s90
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s89
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s88
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s53
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s77
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, vcc_lo
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s76
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s62
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s30
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s61
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s52
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s58
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s31
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s57
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, vcc_hi
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s47
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s35
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s46
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s51
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s45
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s36
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s44
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s34
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s14
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s38
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s12
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s50
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s39
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s37
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s8
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s48
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s49
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
; SI-NEXT: v_readlane_b32 s30, v45, 34
; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
; SI-NEXT: v_readlane_b32 s31, v45, 35
@@ -173313,13 +173490,13 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s4, v21
; SI-NEXT: v_writelane_b32 v56, s4, 21
; SI-NEXT: v_readfirstlane_b32 s4, v20
+; SI-NEXT: v_writelane_b32 v56, s4, 22
+; SI-NEXT: v_readfirstlane_b32 s4, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:332
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:328
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:324
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:320
-; SI-NEXT: v_writelane_b32 v56, s4, 22
-; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:316
; SI-NEXT: v_writelane_b32 v56, s4, 23
; SI-NEXT: v_readfirstlane_b32 s4, v18
; SI-NEXT: v_writelane_b32 v56, s4, 24
@@ -173329,28 +173506,26 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_writelane_b32 v56, s4, 26
; SI-NEXT: v_readfirstlane_b32 s4, v15
; SI-NEXT: v_writelane_b32 v56, s4, 27
-; SI-NEXT: v_readfirstlane_b32 s4, v14
-; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v11
-; SI-NEXT: v_writelane_b32 v56, s4, 29
+; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v9
-; SI-NEXT: v_writelane_b32 v56, s4, 30
+; SI-NEXT: v_writelane_b32 v56, s4, 29
; SI-NEXT: v_readfirstlane_b32 s4, v8
-; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_writelane_b32 v56, s4, 30
; SI-NEXT: v_readfirstlane_b32 s4, v7
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:288
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:284
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:264
-; SI-NEXT: v_writelane_b32 v56, s4, 32
-; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_readfirstlane_b32 s87, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:280
; SI-NEXT: v_readfirstlane_b32 s98, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276
; SI-NEXT: v_readfirstlane_b32 s99, v4
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272
-; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268
+; SI-NEXT: v_readfirstlane_b32 s86, v14
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:312
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:308
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292
@@ -173360,34 +173535,35 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
; SI-NEXT: v_readfirstlane_b32 s76, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:296
-; SI-NEXT: v_writelane_b32 v56, s4, 34
+; SI-NEXT: v_writelane_b32 v56, s4, 32
; SI-NEXT: v_readfirstlane_b32 s77, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260
; SI-NEXT: ; implicit-def: $vgpr47 : SGPR spill to VGPR lane
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v22
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 s81, v21
+; SI-NEXT: v_readfirstlane_b32 s83, v21
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:228
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:224
; SI-NEXT: v_readfirstlane_b32 s4, v1
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:252
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:248
-; SI-NEXT: v_writelane_b32 v56, s4, 35
+; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v0
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:244
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
-; SI-NEXT: v_readfirstlane_b32 s82, v23
+; SI-NEXT: v_readfirstlane_b32 s90, v23
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:220
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:216
-; SI-NEXT: v_readfirstlane_b32 s80, v20
+; SI-NEXT: v_readfirstlane_b32 s82, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:232
-; SI-NEXT: v_readfirstlane_b32 s83, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:232
+; SI-NEXT: v_readfirstlane_b32 s94, v20
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:208
-; SI-NEXT: v_writelane_b32 v56, s4, 36
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:208
+; SI-NEXT: v_writelane_b32 v47, s94, 0
+; SI-NEXT: v_writelane_b32 v56, s4, 34
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s21, v8
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:156
@@ -173398,56 +173574,56 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s46, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:148
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:144
-; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_readfirstlane_b32 s63, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140
; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:136
-; SI-NEXT: v_readfirstlane_b32 s73, v5
-; SI-NEXT: v_readfirstlane_b32 s40, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:128
+; SI-NEXT: v_readfirstlane_b32 s40, v4
; SI-NEXT: v_readfirstlane_b32 s69, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
; SI-NEXT: v_readfirstlane_b32 s47, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
; SI-NEXT: v_readfirstlane_b32 s55, v9
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:164
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
-; SI-NEXT: v_readfirstlane_b32 s57, v14
+; SI-NEXT: v_readfirstlane_b32 s56, v14
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:192
; SI-NEXT: v_readfirstlane_b32 s16, v12
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
; SI-NEXT: v_readfirstlane_b32 s54, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:172
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
-; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:180
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:176
-; SI-NEXT: v_readfirstlane_b32 s60, v24
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v21
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 s59, v24
+; SI-NEXT: v_readfirstlane_b32 s62, v21
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:28
; SI-NEXT: s_waitcnt expcnt(3)
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
-; SI-NEXT: v_readfirstlane_b32 s84, v17
+; SI-NEXT: v_readfirstlane_b32 s74, v17
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
-; SI-NEXT: v_readfirstlane_b32 s90, v18
+; SI-NEXT: v_readfirstlane_b32 s29, v18
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
-; SI-NEXT: v_readfirstlane_b32 s23, v25
+; SI-NEXT: v_readfirstlane_b32 s72, v25
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:20
; SI-NEXT: s_waitcnt expcnt(2)
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:16
-; SI-NEXT: v_readfirstlane_b32 s65, v23
+; SI-NEXT: v_readfirstlane_b32 s64, v23
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:12
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:8
@@ -173457,190 +173633,187 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s37, v22
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:52
; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
-; SI-NEXT: v_readfirstlane_b32 s56, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:44
+; SI-NEXT: v_readfirstlane_b32 s71, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
-; SI-NEXT: v_readfirstlane_b32 s17, v16
+; SI-NEXT: v_readfirstlane_b32 s58, v16
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
-; SI-NEXT: v_readfirstlane_b32 s62, v1
+; SI-NEXT: v_readfirstlane_b32 s60, v1
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:80
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:36
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
-; SI-NEXT: v_readfirstlane_b32 s94, v29
+; SI-NEXT: v_readfirstlane_b32 s65, v29
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32
-; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s13, v11
-; SI-NEXT: v_readfirstlane_b32 s66, v33
+; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_readfirstlane_b32 s66, v33
; SI-NEXT: v_readfirstlane_b32 s14, v34
; SI-NEXT: v_readfirstlane_b32 s15, v6
; SI-NEXT: v_readfirstlane_b32 s93, v39
-; SI-NEXT: v_readfirstlane_b32 s39, v35
-; SI-NEXT: v_readfirstlane_b32 s49, v4
-; SI-NEXT: v_readfirstlane_b32 s51, v37
-; SI-NEXT: v_writelane_b32 v47, s51, 0
-; SI-NEXT: v_writelane_b32 v47, s54, 1
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s12, v38
-; SI-NEXT: v_writelane_b32 v47, s83, 2
+; SI-NEXT: v_writelane_b32 v47, s12, 1
+; SI-NEXT: v_readfirstlane_b32 s57, v26
; SI-NEXT: v_readfirstlane_b32 s52, v9
-; SI-NEXT: v_writelane_b32 v47, s12, 3
-; SI-NEXT: v_writelane_b32 v47, s52, 4
+; SI-NEXT: v_writelane_b32 v47, s52, 2
; SI-NEXT: v_readfirstlane_b32 s67, v32
-; SI-NEXT: v_readfirstlane_b32 s58, v14
-; SI-NEXT: v_readfirstlane_b32 s59, v26
-; SI-NEXT: v_readfirstlane_b32 s61, v27
+; SI-NEXT: v_readfirstlane_b32 s80, v14
+; SI-NEXT: v_readfirstlane_b32 s17, v27
+; SI-NEXT: v_readfirstlane_b32 s61, v12
+; SI-NEXT: v_readfirstlane_b32 s73, v28
; SI-NEXT: v_readfirstlane_b32 s53, v31
-; SI-NEXT: v_writelane_b32 v47, s53, 5
-; SI-NEXT: v_writelane_b32 v47, s55, 6
-; SI-NEXT: v_writelane_b32 v47, s13, 7
-; SI-NEXT: v_writelane_b32 v47, s67, 8
-; SI-NEXT: v_writelane_b32 v47, s88, 9
-; SI-NEXT: v_writelane_b32 v47, s44, 10
-; SI-NEXT: v_writelane_b32 v47, s45, 11
-; SI-NEXT: v_writelane_b32 v47, s46, 12
-; SI-NEXT: v_writelane_b32 v47, s78, 13
-; SI-NEXT: v_writelane_b32 v47, s66, 14
-; SI-NEXT: v_writelane_b32 v47, s47, 15
-; SI-NEXT: v_writelane_b32 v47, s56, 16
-; SI-NEXT: v_writelane_b32 v47, s57, 17
-; SI-NEXT: v_writelane_b32 v47, s14, 18
-; SI-NEXT: v_writelane_b32 v47, s58, 19
-; SI-NEXT: v_writelane_b32 v47, s59, 20
-; SI-NEXT: v_writelane_b32 v47, s17, 21
-; SI-NEXT: v_writelane_b32 v47, s60, 22
-; SI-NEXT: v_writelane_b32 v47, s15, 23
-; SI-NEXT: v_writelane_b32 v47, s61, 24
-; SI-NEXT: v_readfirstlane_b32 s63, v12
-; SI-NEXT: v_writelane_b32 v47, s62, 25
-; SI-NEXT: v_writelane_b32 v47, s63, 26
-; SI-NEXT: v_writelane_b32 v47, s16, 27
+; SI-NEXT: v_writelane_b32 v47, s53, 3
+; SI-NEXT: v_writelane_b32 v47, s55, 4
+; SI-NEXT: v_writelane_b32 v47, s13, 5
+; SI-NEXT: v_writelane_b32 v47, s67, 6
+; SI-NEXT: v_writelane_b32 v47, s88, 7
+; SI-NEXT: v_writelane_b32 v47, s44, 8
+; SI-NEXT: v_writelane_b32 v47, s45, 9
+; SI-NEXT: v_writelane_b32 v47, s46, 10
+; SI-NEXT: v_writelane_b32 v47, s78, 11
+; SI-NEXT: v_writelane_b32 v47, s66, 12
+; SI-NEXT: v_writelane_b32 v47, s71, 13
+; SI-NEXT: v_writelane_b32 v47, s14, 14
+; SI-NEXT: v_writelane_b32 v47, s80, 15
+; SI-NEXT: v_writelane_b32 v47, s57, 16
+; SI-NEXT: v_writelane_b32 v47, s58, 17
+; SI-NEXT: v_writelane_b32 v47, s59, 18
+; SI-NEXT: v_writelane_b32 v47, s15, 19
+; SI-NEXT: v_writelane_b32 v47, s17, 20
+; SI-NEXT: v_writelane_b32 v47, s60, 21
; SI-NEXT: v_readfirstlane_b32 s92, v2
-; SI-NEXT: v_writelane_b32 v47, vcc_lo, 28
-; SI-NEXT: v_writelane_b32 v47, s72, 29
-; SI-NEXT: v_writelane_b32 v47, s73, 30
-; SI-NEXT: v_writelane_b32 v47, s23, 31
-; SI-NEXT: v_readfirstlane_b32 s74, v13
+; SI-NEXT: v_writelane_b32 v47, s61, 22
+; SI-NEXT: v_writelane_b32 v47, s16, 23
+; SI-NEXT: v_writelane_b32 v47, s62, 24
+; SI-NEXT: v_writelane_b32 v47, s63, 25
; SI-NEXT: v_readfirstlane_b32 s95, v48
-; SI-NEXT: v_writelane_b32 v47, s18, 32
-; SI-NEXT: v_readfirstlane_b32 s64, v28
-; SI-NEXT: v_writelane_b32 v47, s74, 33
-; SI-NEXT: v_writelane_b32 v47, s64, 34
+; SI-NEXT: v_writelane_b32 v47, vcc_lo, 26
+; SI-NEXT: v_writelane_b32 v47, s72, 27
+; SI-NEXT: v_readfirstlane_b32 s23, v13
+; SI-NEXT: v_writelane_b32 v47, s18, 28
+; SI-NEXT: v_writelane_b32 v47, s23, 29
+; SI-NEXT: v_writelane_b32 v47, s73, 30
; SI-NEXT: v_readfirstlane_b32 s41, v30
+; SI-NEXT: v_writelane_b32 v47, s40, 31
+; SI-NEXT: v_writelane_b32 v47, s41, 32
+; SI-NEXT: v_readfirstlane_b32 s68, v10
+; SI-NEXT: v_readfirstlane_b32 s38, v5
+; SI-NEXT: v_writelane_b32 v47, s74, 33
+; SI-NEXT: v_readfirstlane_b32 s39, v35
+; SI-NEXT: v_writelane_b32 v47, s68, 34
+; SI-NEXT: v_writelane_b32 v47, s64, 35
; SI-NEXT: v_readfirstlane_b32 s19, v41
-; SI-NEXT: v_writelane_b32 v56, s19, 37
+; SI-NEXT: v_writelane_b32 v56, s19, 35
; SI-NEXT: v_readfirstlane_b32 s43, v51
; SI-NEXT: v_readfirstlane_b32 s42, v18
-; SI-NEXT: v_writelane_b32 v56, s42, 38
-; SI-NEXT: v_writelane_b32 v56, s43, 39
+; SI-NEXT: v_writelane_b32 v56, s42, 36
+; SI-NEXT: v_writelane_b32 v56, s43, 37
; SI-NEXT: v_readfirstlane_b32 s6, v42
; SI-NEXT: v_readfirstlane_b32 s7, v23
-; SI-NEXT: v_writelane_b32 v56, s6, 40
+; SI-NEXT: v_writelane_b32 v56, s6, 38
; SI-NEXT: v_readfirstlane_b32 s75, v52
-; SI-NEXT: v_writelane_b32 v56, s7, 41
+; SI-NEXT: v_writelane_b32 v56, s7, 39
; SI-NEXT: s_waitcnt vmcnt(13)
; SI-NEXT: v_readfirstlane_b32 s79, v0
-; SI-NEXT: v_writelane_b32 v56, s75, 42
+; SI-NEXT: v_writelane_b32 v56, s75, 40
; SI-NEXT: s_waitcnt vmcnt(11)
; SI-NEXT: v_readfirstlane_b32 s89, v22
-; SI-NEXT: v_writelane_b32 v56, s79, 43
+; SI-NEXT: v_writelane_b32 v56, s79, 41
; SI-NEXT: v_readfirstlane_b32 s91, v53
-; SI-NEXT: v_writelane_b32 v56, s89, 44
-; SI-NEXT: v_writelane_b32 v56, s91, 45
-; SI-NEXT: v_writelane_b32 v56, s92, 46
+; SI-NEXT: v_writelane_b32 v56, s89, 42
+; SI-NEXT: v_writelane_b32 v56, s91, 43
+; SI-NEXT: v_writelane_b32 v56, s92, 44
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_readfirstlane_b32 s8, v54
-; SI-NEXT: v_writelane_b32 v56, s93, 47
+; SI-NEXT: v_writelane_b32 v56, s93, 45
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s9, v20
-; SI-NEXT: v_writelane_b32 v56, s8, 48
-; SI-NEXT: v_writelane_b32 v56, s9, 49
+; SI-NEXT: v_readfirstlane_b32 s9, v19
+; SI-NEXT: v_writelane_b32 v56, s8, 46
+; SI-NEXT: v_writelane_b32 v56, s9, 47
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s31, v16
-; SI-NEXT: v_writelane_b32 v56, s95, 50
-; SI-NEXT: v_writelane_b32 v56, s31, 51
+; SI-NEXT: v_writelane_b32 v56, s95, 48
+; SI-NEXT: v_writelane_b32 v56, s31, 49
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_writelane_b32 v56, s80, 52
-; SI-NEXT: v_writelane_b32 v47, s40, 35
+; SI-NEXT: v_writelane_b32 v56, s82, 50
; SI-NEXT: v_readfirstlane_b32 s36, v49
-; SI-NEXT: v_writelane_b32 v56, s34, 53
-; SI-NEXT: v_writelane_b32 v47, s41, 36
-; SI-NEXT: v_readfirstlane_b32 s68, v10
-; SI-NEXT: v_writelane_b32 v56, s36, 54
-; SI-NEXT: v_writelane_b32 v47, s84, 37
+; SI-NEXT: v_writelane_b32 v56, s34, 51
+; SI-NEXT: v_writelane_b32 v56, s36, 52
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_readfirstlane_b32 s10, v50
-; SI-NEXT: v_writelane_b32 v56, s81, 55
-; SI-NEXT: v_writelane_b32 v47, s68, 38
-; SI-NEXT: v_readfirstlane_b32 s38, v5
-; SI-NEXT: v_writelane_b32 v56, s10, 56
-; SI-NEXT: v_writelane_b32 v47, s65, 39
-; SI-NEXT: v_writelane_b32 v56, s38, 57
-; SI-NEXT: v_writelane_b32 v47, s90, 40
+; SI-NEXT: v_writelane_b32 v56, s83, 53
+; SI-NEXT: v_writelane_b32 v56, s10, 54
+; SI-NEXT: v_writelane_b32 v56, s38, 55
; SI-NEXT: v_readfirstlane_b32 s11, v17
-; SI-NEXT: v_writelane_b32 v56, s39, 58
-; SI-NEXT: v_writelane_b32 v47, s69, 41
+; SI-NEXT: v_writelane_b32 v56, s39, 56
; SI-NEXT: v_readfirstlane_b32 s48, v36
-; SI-NEXT: v_writelane_b32 v56, s11, 59
-; SI-NEXT: v_writelane_b32 v47, s94, 42
-; SI-NEXT: v_writelane_b32 v56, s48, 60
-; SI-NEXT: v_writelane_b32 v47, s70, 43
-; SI-NEXT: v_readfirstlane_b32 s35, v19
-; SI-NEXT: v_writelane_b32 v56, s49, 61
-; SI-NEXT: v_writelane_b32 v47, s37, 44
-; SI-NEXT: v_readfirstlane_b32 s71, v15
+; SI-NEXT: v_writelane_b32 v56, s11, 57
+; SI-NEXT: v_writelane_b32 v47, s29, 36
+; SI-NEXT: v_readfirstlane_b32 s49, v4
+; SI-NEXT: v_writelane_b32 v56, s48, 58
+; SI-NEXT: v_writelane_b32 v47, s69, 37
+; SI-NEXT: v_writelane_b32 v56, s49, 59
+; SI-NEXT: v_writelane_b32 v47, s65, 38
; SI-NEXT: v_readfirstlane_b32 s50, v3
+; SI-NEXT: v_writelane_b32 v56, s90, 60
+; SI-NEXT: v_writelane_b32 v47, s70, 39
+; SI-NEXT: v_readfirstlane_b32 s35, v20
+; SI-NEXT: v_readfirstlane_b32 s51, v37
+; SI-NEXT: v_writelane_b32 v56, s50, 61
+; SI-NEXT: v_writelane_b32 v47, s37, 40
+; SI-NEXT: v_readfirstlane_b32 s81, v15
; SI-NEXT: v_readfirstlane_b32 s22, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s85, v24
+; SI-NEXT: v_readfirstlane_b32 s30, v24
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s25, v40
; SI-NEXT: v_readfirstlane_b32 s28, v21
-; SI-NEXT: v_readfirstlane_b32 s87, v25
+; SI-NEXT: v_readfirstlane_b32 s85, v25
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v43
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_readfirstlane_b32 s26, v29
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s27, v44
-; SI-NEXT: v_writelane_b32 v56, s82, 62
-; SI-NEXT: v_writelane_b32 v47, s35, 45
-; SI-NEXT: v_writelane_b32 v56, s50, 63
-; SI-NEXT: v_writelane_b32 v47, s71, 46
+; SI-NEXT: v_writelane_b32 v56, s51, 62
+; SI-NEXT: v_writelane_b32 v47, s35, 41
+; SI-NEXT: v_writelane_b32 v56, s54, 63
+; SI-NEXT: v_writelane_b32 v47, s81, 42
; SI-NEXT: s_cbranch_scc0 .LBB93_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: v_readlane_b32 s4, v56, 11
; SI-NEXT: v_readlane_b32 s5, v56, 10
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: v_writelane_b32 v47, s21, 47
+; SI-NEXT: v_writelane_b32 v47, s21, 57
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 4
-; SI-NEXT: v_writelane_b32 v47, s4, 48
+; SI-NEXT: v_writelane_b32 v47, s4, 43
; SI-NEXT: s_and_b32 s4, s24, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 34
-; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_readlane_b32 s5, v56, 32
+; SI-NEXT: v_writelane_b32 v47, s4, 45
; SI-NEXT: s_and_b32 s4, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 29
-; SI-NEXT: v_writelane_b32 v47, s4, 50
+; SI-NEXT: v_readlane_b32 s5, v56, 28
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s4, 49
; SI-NEXT: v_readlane_b32 s4, v56, 24
; SI-NEXT: s_and_b32 s5, s4, 0xff
; SI-NEXT: v_readlane_b32 s4, v56, 23
; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_writelane_b32 v47, s4, 52
+; SI-NEXT: v_writelane_b32 v47, s4, 51
; SI-NEXT: v_readlane_b32 s4, v56, 16
; SI-NEXT: v_readlane_b32 s5, v56, 15
; SI-NEXT: s_and_b32 s4, s4, 0xff
@@ -173650,41 +173823,45 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s4, s7, 0xff
; SI-NEXT: s_lshl_b32 s5, s6, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 54
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_and_b32 s4, s9, 0xff
; SI-NEXT: s_lshl_b32 s5, s8, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s4, 58
; SI-NEXT: s_and_b32 s4, s11, 0xff
; SI-NEXT: s_lshl_b32 s5, s10, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 56
+; SI-NEXT: v_writelane_b32 v47, s4, 59
; SI-NEXT: s_and_b32 s4, s13, 0xff
; SI-NEXT: s_lshl_b32 s5, s12, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 57
+; SI-NEXT: v_writelane_b32 v47, s4, 60
; SI-NEXT: s_and_b32 s4, s15, 0xff
; SI-NEXT: s_lshl_b32 s5, s14, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 58
+; SI-NEXT: v_writelane_b32 v47, s4, 61
; SI-NEXT: s_and_b32 s4, s68, 0xff
; SI-NEXT: s_lshl_b32 s5, s41, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 59
-; SI-NEXT: s_and_b32 s4, s71, 0xff
+; SI-NEXT: v_writelane_b32 v47, s4, 62
+; SI-NEXT: s_and_b32 s4, s81, 0xff
; SI-NEXT: s_lshl_b32 s5, s35, 8
-; SI-NEXT: s_or_b32 s71, s4, s5
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v47, s4, 63
; SI-NEXT: s_and_b32 s4, s37, 0xff
; SI-NEXT: s_lshl_b32 s5, s70, 8
-; SI-NEXT: s_or_b32 s70, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_lshl_b32 s5, s40, 8
-; SI-NEXT: s_or_b32 s69, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 0
+; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 1
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s16, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 7
-; SI-NEXT: v_writelane_b32 v47, s4, 60
+; SI-NEXT: v_writelane_b32 v46, s4, 2
; SI-NEXT: s_and_b32 s4, s20, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s5, s4, s5
@@ -173693,57 +173870,52 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readlane_b32 s6, v56, 5
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_or_b32 s7, s6, s4
+; SI-NEXT: s_or_b32 s81, s6, s4
; SI-NEXT: v_readlane_b32 s4, v56, 9
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: v_readlane_b32 s6, v56, 8
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s7, 61
; SI-NEXT: s_or_b32 s4, s6, s4
-; SI-NEXT: s_or_b32 s5, s5, s7
; SI-NEXT: v_readlane_b32 s6, v56, 1
; SI-NEXT: v_readlane_b32 s7, v56, 0
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_or_b32 s7, s6, s7
-; SI-NEXT: v_readlane_b32 s6, v56, 36
+; SI-NEXT: v_readlane_b32 s6, v56, 34
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 35
+; SI-NEXT: v_readlane_b32 s8, v56, 33
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_or_b32 s9, s8, s6
; SI-NEXT: v_readlane_b32 s6, v56, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: v_readlane_b32 s8, v56, 2
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s9, 3
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s9, 62
-; SI-NEXT: s_or_b32 s6, s8, s6
; SI-NEXT: s_or_b32 s7, s7, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 33
-; SI-NEXT: v_readlane_b32 s9, v56, 32
-; SI-NEXT: s_and_b32 s8, s8, 0xff
+; SI-NEXT: v_readlane_b32 s9, v56, 31
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_and_b32 s8, s87, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
; SI-NEXT: s_or_b32 s9, s8, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 31
+; SI-NEXT: v_readlane_b32 s8, v56, 30
; SI-NEXT: s_and_b32 s8, s8, 0xff
-; SI-NEXT: v_readlane_b32 s10, v56, 30
+; SI-NEXT: v_readlane_b32 s10, v56, 29
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s10, 24
; SI-NEXT: s_or_b32 s11, s10, s8
; SI-NEXT: s_and_b32 s8, s99, 0xff
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s11, 4
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s98, 24
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s11, 63
-; SI-NEXT: s_or_b32 s8, s10, s8
; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: v_readlane_b32 s10, v56, 28
; SI-NEXT: v_readlane_b32 s11, v56, 27
-; SI-NEXT: s_and_b32 s10, s10, 0xff
+; SI-NEXT: s_or_b32 s8, s10, s8
+; SI-NEXT: s_and_b32 s10, s86, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 8
; SI-NEXT: s_or_b32 s11, s10, s11
; SI-NEXT: v_readlane_b32 s10, v56, 26
@@ -173751,16 +173923,12 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readlane_b32 s12, v56, 25
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s13, s12, s10
+; SI-NEXT: s_or_b32 s84, s12, s10
; SI-NEXT: s_and_b32 s10, s97, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s96, 24
-; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: v_readlane_b32 s12, v56, 20
-; SI-NEXT: v_writelane_b32 v46, s13, 0
-; SI-NEXT: s_or_b32 s11, s11, s13
; SI-NEXT: s_and_b32 s13, s12, 0xff
; SI-NEXT: v_readlane_b32 s12, v56, 19
; SI-NEXT: s_lshl_b32 s12, s12, 8
@@ -173770,7 +173938,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s14, s12, 16
; SI-NEXT: v_readlane_b32 s12, v56, 17
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s86, s12, s14
+; SI-NEXT: s_or_b32 s69, s12, s14
; SI-NEXT: v_readlane_b32 s12, v56, 22
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s14, s12, 16
@@ -173784,7 +173952,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s14, s26, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_lshl_b32 s16, vcc_hi, 24
-; SI-NEXT: s_or_b32 s29, s16, s14
+; SI-NEXT: s_or_b32 s70, s16, s14
; SI-NEXT: v_readlane_b32 s14, v56, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s16, s14, 16
@@ -173794,16 +173962,16 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s16, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s25, 8
; SI-NEXT: s_or_b32 s18, s16, s18
-; SI-NEXT: s_and_b32 s16, s85, 0xff
-; SI-NEXT: v_writelane_b32 v46, s20, 1
+; SI-NEXT: s_and_b32 s16, s30, 0xff
+; SI-NEXT: v_writelane_b32 v46, s20, 5
; SI-NEXT: s_lshl_b32 s16, s16, 16
; SI-NEXT: s_lshl_b32 s20, s22, 24
-; SI-NEXT: v_writelane_b32 v46, vcc_hi, 2
-; SI-NEXT: v_writelane_b32 v46, s28, 3
+; SI-NEXT: v_writelane_b32 v46, vcc_hi, 6
+; SI-NEXT: v_writelane_b32 v46, s28, 7
; SI-NEXT: s_or_b32 s28, s20, s16
-; SI-NEXT: s_and_b32 s20, s87, 0xff
+; SI-NEXT: s_and_b32 s20, s85, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: v_writelane_b32 v46, s24, 4
+; SI-NEXT: v_writelane_b32 v46, s24, 8
; SI-NEXT: s_lshl_b32 s24, s19, 24
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s40, s24, s20
@@ -173817,7 +173985,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s19, s24, s18
; SI-NEXT: s_and_b32 s24, s89, 0xff
; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: v_writelane_b32 v46, s25, 5
+; SI-NEXT: v_writelane_b32 v46, s25, 9
; SI-NEXT: s_lshl_b32 s25, s91, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s42, s25, s24
@@ -173842,7 +174010,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s25, s38, 0xff
; SI-NEXT: s_lshl_b32 s25, s25, 16
; SI-NEXT: s_lshl_b32 s26, s39, 24
-; SI-NEXT: v_writelane_b32 v46, s27, 6
+; SI-NEXT: v_writelane_b32 v46, s27, 10
; SI-NEXT: s_or_b32 s27, s26, s25
; SI-NEXT: s_and_b32 s26, s50, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
@@ -173863,44 +174031,44 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s38, s75, s26
; SI-NEXT: s_or_b32 s39, s20, s18
-; SI-NEXT: s_and_b32 s20, s63, 0xff
-; SI-NEXT: s_lshl_b32 s26, s61, 8
+; SI-NEXT: s_and_b32 s20, s61, 0xff
+; SI-NEXT: s_lshl_b32 s26, s17, 8
; SI-NEXT: s_or_b32 s20, s20, s26
-; SI-NEXT: s_and_b32 s26, s58, 0xff
+; SI-NEXT: s_and_b32 s26, s80, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
-; SI-NEXT: s_lshl_b32 s75, s59, 24
+; SI-NEXT: s_lshl_b32 s75, s57, 24
; SI-NEXT: s_or_b32 s26, s75, s26
-; SI-NEXT: s_and_b32 s75, s74, 0xff
+; SI-NEXT: s_and_b32 s75, s23, 0xff
; SI-NEXT: s_lshl_b32 s75, s75, 16
-; SI-NEXT: s_lshl_b32 s78, s64, 24
+; SI-NEXT: s_lshl_b32 s78, s73, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s48, s78, s75
; SI-NEXT: s_or_b32 s49, s20, s26
-; SI-NEXT: s_and_b32 s20, s23, 0xff
-; SI-NEXT: s_lshl_b32 s75, vcc_lo, 8
+; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_lshl_b32 s75, s62, 8
; SI-NEXT: s_or_b32 s75, s20, s75
-; SI-NEXT: s_and_b32 s20, s60, 0xff
+; SI-NEXT: s_and_b32 s20, s59, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_lshl_b32 s78, s56, 24
+; SI-NEXT: s_lshl_b32 s78, s71, 24
; SI-NEXT: s_or_b32 s20, s78, s20
-; SI-NEXT: s_and_b32 s78, s94, 0xff
+; SI-NEXT: s_and_b32 s78, s65, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
-; SI-NEXT: s_lshl_b32 s79, s65, 24
+; SI-NEXT: s_lshl_b32 s79, s64, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_or_b32 s50, s79, s78
; SI-NEXT: s_or_b32 s51, s75, s20
-; SI-NEXT: s_and_b32 s75, s62, 0xff
-; SI-NEXT: s_lshl_b32 s78, s17, 8
+; SI-NEXT: s_and_b32 s75, s60, 0xff
+; SI-NEXT: s_lshl_b32 s78, s58, 8
; SI-NEXT: s_or_b32 s75, s75, s78
; SI-NEXT: s_and_b32 s78, s45, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
; SI-NEXT: s_lshl_b32 s79, s44, 24
; SI-NEXT: s_or_b32 s78, s79, s78
-; SI-NEXT: s_and_b32 s79, s90, 0xff
+; SI-NEXT: s_and_b32 s79, s29, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s88, s84, 24
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 47
+; SI-NEXT: v_readlane_b32 s16, v47, 57
; SI-NEXT: s_or_b32 s52, s88, s79
; SI-NEXT: s_or_b32 s53, s75, s78
; SI-NEXT: s_and_b32 s75, s46, 0xff
@@ -173910,167 +174078,188 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_lshl_b32 s88, s54, 24
; SI-NEXT: s_or_b32 s88, s88, s79
-; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_and_b32 s79, vcc_lo, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s72, 24
+; SI-NEXT: s_lshl_b32 s89, s63, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 48
+; SI-NEXT: v_readlane_b32 s16, v47, 43
; SI-NEXT: s_or_b32 s54, s89, s79
; SI-NEXT: s_or_b32 s55, s75, s88
-; SI-NEXT: s_and_b32 s75, s83, 0xff
-; SI-NEXT: s_lshl_b32 s79, s82, 8
+; SI-NEXT: s_and_b32 s75, s94, 0xff
+; SI-NEXT: s_lshl_b32 s79, s90, 8
; SI-NEXT: s_and_b32 s44, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 49
+; SI-NEXT: v_readlane_b32 s16, v47, 45
; SI-NEXT: s_or_b32 s75, s75, s79
-; SI-NEXT: s_and_b32 s79, s81, 0xff
+; SI-NEXT: s_and_b32 s79, s83, 0xff
; SI-NEXT: s_and_b32 s45, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 50
+; SI-NEXT: v_readlane_b32 s16, v47, 47
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s80, 24
+; SI-NEXT: s_lshl_b32 s89, s82, 24
; SI-NEXT: s_and_b32 s46, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: v_readlane_b32 s16, v47, 49
; SI-NEXT: s_or_b32 s90, s89, s79
+; SI-NEXT: s_mov_b32 s71, s47
; SI-NEXT: s_lshl_b32 s89, s47, 24
; SI-NEXT: s_and_b32 s47, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 52
-; SI-NEXT: s_and_b32 s79, s57, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: s_and_b32 s79, s56, 0xff
; SI-NEXT: s_and_b32 s56, s16, 0xffff
; SI-NEXT: v_readlane_b32 s16, v47, 53
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_and_b32 s57, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 54
+; SI-NEXT: v_readlane_b32 s16, v47, 55
; SI-NEXT: s_or_b32 s64, s89, s79
; SI-NEXT: s_and_b32 s79, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 55
+; SI-NEXT: v_readlane_b32 s16, v47, 58
; SI-NEXT: s_and_b32 s89, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 56
+; SI-NEXT: v_readlane_b32 s16, v47, 59
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_and_b32 s17, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 57
+; SI-NEXT: v_readlane_b32 s16, v47, 60
+; SI-NEXT: s_or_b32 s5, s5, s81
+; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_and_b32 s91, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 58
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: s_or_b32 s65, s75, s90
; SI-NEXT: s_and_b32 s92, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 59
-; SI-NEXT: s_or_b32 s13, s13, s86
+; SI-NEXT: v_readlane_b32 s16, v47, 62
+; SI-NEXT: s_or_b32 s74, s44, s4
+; SI-NEXT: s_mov_b32 s75, s5
+; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[4:5], s[6:7], 16
; SI-NEXT: s_and_b32 s93, s16, 0xffff
-; SI-NEXT: s_mov_b32 s16, s85
-; SI-NEXT: s_lshr_b64 s[84:85], s[40:41], 16
-; SI-NEXT: s_and_b32 s94, s71, 0xffff
-; SI-NEXT: s_and_b32 s95, s70, 0xffff
-; SI-NEXT: s_or_b32 s58, s56, s12
-; SI-NEXT: s_mov_b32 s59, s13
-; SI-NEXT: s_lshr_b64 s[70:71], s[12:13], 16
-; SI-NEXT: s_or_b32 s12, s93, s48
-; SI-NEXT: s_mov_b32 s13, s49
-; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
-; SI-NEXT: s_mov_b32 s85, s16
-; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: v_readlane_b32 s16, v47, 63
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 16
+; SI-NEXT: s_or_b32 s11, s11, s84
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: s_lshr_b64 s[4:5], s[10:11], 16
+; SI-NEXT: s_or_b32 s13, s13, s69
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_mov_b32 s25, s22
-; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s22, v47, 60
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: s_lshr_b64 s[4:5], s[12:13], 16
+; SI-NEXT: s_or_b32 s15, s15, s70
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: s_lshr_b64 s[4:5], s[14:15], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: s_lshr_b64 s[4:5], s[40:41], 16
+; SI-NEXT: s_mov_b32 s23, s21
+; SI-NEXT: v_readlane_b32 s21, v46, 1
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: s_and_b32 s94, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_and_b32 vcc_lo, s21, 0xffff
+; SI-NEXT: v_readlane_b32 s21, v46, 2
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: s_lshr_b64 s[4:5], s[42:43], 16
+; SI-NEXT: s_and_b32 s95, s16, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s21, 0xffff
; SI-NEXT: s_or_b32 s60, s47, s10
; SI-NEXT: s_mov_b32 s61, s11
-; SI-NEXT: s_lshr_b64 vcc, s[10:11], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_or_b32 s10, s94, s50
; SI-NEXT: s_mov_b32 s11, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[50:51], 16
-; SI-NEXT: s_lshr_b32 s49, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 62
-; SI-NEXT: s_or_b32 s15, s15, s29
-; SI-NEXT: s_mov_b32 s23, s21
-; SI-NEXT: s_or_b32 s65, s75, s90
-; SI-NEXT: s_and_b32 s21, s69, 0xffff
-; SI-NEXT: s_and_b32 s22, s22, 0xffff
-; SI-NEXT: s_or_b32 s62, s46, s8
-; SI-NEXT: s_mov_b32 s63, s9
-; SI-NEXT: s_lshr_b64 s[66:67], s[8:9], 16
-; SI-NEXT: s_or_b32 s8, s95, s52
-; SI-NEXT: s_mov_b32 s9, s53
-; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
-; SI-NEXT: s_lshr_b32 s51, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 63
-; SI-NEXT: s_or_b32 s74, s44, s4
-; SI-NEXT: s_mov_b32 s75, s5
-; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: v_readlane_b32 s16, v46, 3
; SI-NEXT: s_or_b32 s72, s45, s6
; SI-NEXT: s_mov_b32 s73, s7
-; SI-NEXT: s_lshr_b64 s[30:31], s[6:7], 16
+; SI-NEXT: s_or_b32 s62, s46, s8
+; SI-NEXT: s_mov_b32 s63, s9
+; SI-NEXT: s_or_b32 s58, s56, s12
+; SI-NEXT: s_mov_b32 s59, s13
; SI-NEXT: s_or_b32 s56, s57, s14
; SI-NEXT: s_mov_b32 s57, s15
-; SI-NEXT: s_lshr_b64 s[80:81], s[14:15], 16
-; SI-NEXT: s_or_b32 s44, s89, s42
-; SI-NEXT: s_mov_b32 s45, s43
-; SI-NEXT: s_lshr_b64 s[68:69], s[42:43], 16
-; SI-NEXT: s_or_b32 s42, s17, s34
-; SI-NEXT: s_mov_b32 s43, s35
-; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
+; SI-NEXT: v_writelane_b32 v47, s5, 56
; SI-NEXT: s_or_b32 s14, s92, s38
; SI-NEXT: s_mov_b32 s15, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[38:39], 16
-; SI-NEXT: s_or_b32 s6, s21, s54
+; SI-NEXT: s_or_b32 s12, s93, s48
+; SI-NEXT: s_mov_b32 s13, s49
+; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
+; SI-NEXT: s_or_b32 s8, s95, s52
+; SI-NEXT: s_mov_b32 s9, s53
+; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
+; SI-NEXT: s_or_b32 s6, vcc_lo, s54
; SI-NEXT: s_mov_b32 s7, s55
; SI-NEXT: s_lshr_b64 s[54:55], s[54:55], 16
-; SI-NEXT: s_or_b32 s4, s22, s64
+; SI-NEXT: s_or_b32 s4, vcc_hi, s64
; SI-NEXT: s_mov_b32 s5, s65
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 16
-; SI-NEXT: s_lshr_b32 s53, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_lshr_b32 s51, s16, 16
+; SI-NEXT: v_readlane_b32 s16, v46, 4
; SI-NEXT: s_or_b32 s46, s79, s40
; SI-NEXT: s_mov_b32 s47, s41
+; SI-NEXT: s_or_b32 s44, s89, s42
+; SI-NEXT: s_mov_b32 s45, s43
+; SI-NEXT: s_or_b32 s42, s17, s34
+; SI-NEXT: s_mov_b32 s43, s35
+; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
; SI-NEXT: s_or_b32 s40, s91, s36
; SI-NEXT: s_mov_b32 s41, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[36:37], 16
-; SI-NEXT: s_lshr_b32 s55, s16, 16
-; SI-NEXT: s_lshr_b32 s65, s86, 16
-; SI-NEXT: s_lshr_b32 s35, s29, 16
-; SI-NEXT: s_mov_b32 s94, s66
+; SI-NEXT: s_lshr_b32 s49, s81, 16
+; SI-NEXT: s_lshr_b32 s53, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s84, 16
+; SI-NEXT: s_lshr_b32 s65, s69, 16
+; SI-NEXT: s_lshr_b32 s69, s70, 16
; SI-NEXT: s_lshr_b32 s79, s28, 16
-; SI-NEXT: v_readlane_b32 s28, v46, 3
-; SI-NEXT: v_readlane_b32 vcc_hi, v46, 2
+; SI-NEXT: v_readlane_b32 s28, v46, 7
+; SI-NEXT: v_readlane_b32 vcc_hi, v46, 6
; SI-NEXT: s_lshr_b32 s89, s19, 16
; SI-NEXT: s_lshr_b32 s91, s24, 16
-; SI-NEXT: v_readlane_b32 s24, v46, 4
+; SI-NEXT: v_readlane_b32 s24, v46, 8
; SI-NEXT: s_lshr_b32 s93, s27, 16
-; SI-NEXT: v_readlane_b32 s27, v46, 6
+; SI-NEXT: v_readlane_b32 s27, v46, 10
; SI-NEXT: s_lshr_b32 s95, s18, 16
; SI-NEXT: s_lshr_b32 s31, s26, 16
; SI-NEXT: s_lshr_b32 s17, s20, 16
-; SI-NEXT: v_readlane_b32 s20, v46, 1
+; SI-NEXT: v_readlane_b32 s20, v46, 5
; SI-NEXT: s_lshr_b32 s92, s78, 16
-; SI-NEXT: s_lshr_b32 s78, s88, 16
-; SI-NEXT: v_readlane_b32 s21, v47, 47
+; SI-NEXT: s_lshr_b32 s68, s88, 16
+; SI-NEXT: v_readlane_b32 s21, v47, 57
; SI-NEXT: s_mov_b32 s26, s23
-; SI-NEXT: s_mov_b32 s22, s25
-; SI-NEXT: v_readlane_b32 s25, v46, 5
+; SI-NEXT: v_readlane_b32 s25, v46, 9
; SI-NEXT: s_lshr_b32 s39, s90, 16
; SI-NEXT: s_mov_b64 s[66:67], 0
; SI-NEXT: s_branch .LBB93_3
; SI-NEXT: .LBB93_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s71, s47
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: s_mov_b64 s[66:67], -1
+; SI-NEXT: v_writelane_b32 v47, s4, 47
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr49
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr30
; SI-NEXT: ; implicit-def: $sgpr51
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr70
; SI-NEXT: ; implicit-def: $sgpr65
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr69
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr34
@@ -174092,41 +174281,62 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr92
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr54
-; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s5, 56
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB93_3: ; %Flow
; SI-NEXT: s_and_b64 s[18:19], s[66:67], exec
; SI-NEXT: s_cselect_b32 s16, 1, 0
; SI-NEXT: s_cmp_lg_u32 s16, 1
+; SI-NEXT: v_readlane_b32 s16, v47, 0
+; SI-NEXT: v_readlane_b32 s18, v47, 7
+; SI-NEXT: v_readlane_b32 s19, v47, 8
+; SI-NEXT: v_readlane_b32 s88, v47, 9
+; SI-NEXT: v_readlane_b32 s90, v47, 10
+; SI-NEXT: v_readlane_b32 s78, v47, 11
+; SI-NEXT: v_readlane_b32 s94, v47, 16
+; SI-NEXT: v_readlane_b32 vcc_lo, v47, 27
+; SI-NEXT: v_readlane_b32 s23, v47, 30
+; SI-NEXT: v_readlane_b32 s29, v47, 36
+; SI-NEXT: v_readlane_b32 s37, v47, 40
+; SI-NEXT: v_readlane_b32 s35, v47, 41
+; SI-NEXT: v_readlane_b32 s66, v47, 42
; SI-NEXT: s_cbranch_scc1 .LBB93_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_readlane_b32 s4, v47, 32
+; SI-NEXT: v_readlane_b32 s4, v47, 28
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: v_readlane_b32 s5, v47, 27
-; SI-NEXT: v_readlane_b32 s6, v47, 17
+; SI-NEXT: v_readlane_b32 s5, v47, 23
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_add_i32 s6, s80, 3
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 15
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 24
+; SI-NEXT: s_lshl_b32 s5, s71, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s5, s5, s6
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 2
-; SI-NEXT: s_add_i32 s5, s5, 3
-; SI-NEXT: v_readlane_b32 s6, v56, 62
-; SI-NEXT: v_readlane_b32 s7, v56, 55
+; SI-NEXT: s_add_i32 s5, s16, 3
+; SI-NEXT: v_readlane_b32 s6, v56, 60
+; SI-NEXT: v_readlane_b32 s7, v56, 53
; SI-NEXT: s_and_b32 s5, s5, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 8
; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v56, 52
+; SI-NEXT: v_readlane_b32 s6, v56, 50
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 24
; SI-NEXT: s_lshl_b32 s7, s7, 16
@@ -174134,15 +174344,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v47, 41
+; SI-NEXT: v_readlane_b32 s6, v47, 37
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 35
-; SI-NEXT: v_readlane_b32 s8, v47, 30
+; SI-NEXT: v_readlane_b32 s7, v47, 31
+; SI-NEXT: v_readlane_b32 s8, v47, 26
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_add_i32 s8, s8, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 29
+; SI-NEXT: v_readlane_b32 s7, v47, 25
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_lshl_b32 s8, s8, 16
@@ -174150,14 +174360,13 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s7, s7, s8
; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 12
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 6
+; SI-NEXT: s_add_i32 s7, s90, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 4
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s21, 8
; SI-NEXT: s_add_i32 s9, s9, 3
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 1
+; SI-NEXT: v_readlane_b32 s8, v56, 63
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_lshl_b32 s9, s9, 16
@@ -174165,15 +174374,13 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s8, s8, s9
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 44
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 43
-; SI-NEXT: v_readlane_b32 s10, v47, 40
+; SI-NEXT: s_add_i32 s8, s37, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 39
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
-; SI-NEXT: s_add_i32 s10, s10, 3
+; SI-NEXT: s_add_i32 s10, s29, 3
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 37
+; SI-NEXT: v_readlane_b32 s9, v47, 33
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 24
; SI-NEXT: s_lshl_b32 s10, s10, 16
@@ -174181,31 +174388,27 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_and_b32 s8, s8, 0xffff
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 25
+; SI-NEXT: v_readlane_b32 s9, v47, 21
; SI-NEXT: s_add_i32 s9, s9, 3
-; SI-NEXT: v_readlane_b32 s10, v47, 21
-; SI-NEXT: v_readlane_b32 s11, v47, 11
+; SI-NEXT: v_readlane_b32 s10, v47, 17
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 8
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s11, s88, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 10
; SI-NEXT: s_and_b32 s11, s11, 0xff
-; SI-NEXT: s_lshl_b32 s10, s10, 24
+; SI-NEXT: s_lshl_b32 s10, s19, 24
; SI-NEXT: s_lshl_b32 s11, s11, 16
; SI-NEXT: s_addk_i32 s9, 0x300
; SI-NEXT: s_or_b32 s10, s10, s11
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 46
-; SI-NEXT: s_add_i32 s10, s10, 3
-; SI-NEXT: v_readlane_b32 s11, v47, 45
-; SI-NEXT: v_readlane_b32 s12, v47, 42
+; SI-NEXT: s_add_i32 s10, s66, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 38
; SI-NEXT: s_and_b32 s10, s10, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 8
+; SI-NEXT: s_lshl_b32 s11, s35, 8
; SI-NEXT: s_add_i32 s12, s12, 3
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 39
+; SI-NEXT: v_readlane_b32 s11, v47, 35
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 24
; SI-NEXT: s_lshl_b32 s12, s12, 16
@@ -174213,15 +174416,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s11, s11, s12
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 31
-; SI-NEXT: s_add_i32 s11, s11, 3
-; SI-NEXT: v_readlane_b32 s12, v47, 28
-; SI-NEXT: v_readlane_b32 s13, v47, 22
+; SI-NEXT: s_add_i32 s11, vcc_lo, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 24
+; SI-NEXT: v_readlane_b32 s13, v47, 18
; SI-NEXT: s_and_b32 s11, s11, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 8
; SI-NEXT: s_add_i32 s13, s13, 3
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 16
+; SI-NEXT: v_readlane_b32 s12, v47, 13
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 24
; SI-NEXT: s_lshl_b32 s13, s13, 16
@@ -174229,47 +174431,44 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s12, s12, s13
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 38
+; SI-NEXT: v_readlane_b32 s12, v47, 34
; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: v_readlane_b32 s13, v47, 36
-; SI-NEXT: v_readlane_b32 s14, v47, 33
+; SI-NEXT: v_readlane_b32 s13, v47, 32
+; SI-NEXT: v_readlane_b32 s14, v47, 29
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s13, s13, 8
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 34
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s13, s13, 24
+; SI-NEXT: s_lshl_b32 s13, s23, 24
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_addk_i32 s12, 0x300
; SI-NEXT: s_or_b32 s13, s13, s14
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 26
+; SI-NEXT: v_readlane_b32 s13, v47, 22
; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s14, v47, 24
-; SI-NEXT: v_readlane_b32 s15, v47, 19
+; SI-NEXT: v_readlane_b32 s14, v47, 20
+; SI-NEXT: v_readlane_b32 s15, v47, 15
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 8
; SI-NEXT: s_add_i32 s15, s15, 3
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 20
; SI-NEXT: s_and_b32 s15, s15, 0xff
-; SI-NEXT: s_lshl_b32 s14, s14, 24
+; SI-NEXT: s_lshl_b32 s14, s94, 24
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_addk_i32 s13, 0x300
; SI-NEXT: s_or_b32 s14, s14, s15
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 23
+; SI-NEXT: v_readlane_b32 s14, v47, 19
; SI-NEXT: s_add_i32 s14, s14, 3
-; SI-NEXT: v_readlane_b32 s15, v47, 18
-; SI-NEXT: v_readlane_b32 s16, v47, 13
+; SI-NEXT: v_readlane_b32 s15, v47, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 8
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s78, 3
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 14
+; SI-NEXT: v_readlane_b32 s15, v47, 12
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 24
; SI-NEXT: s_lshl_b32 s16, s16, 16
@@ -174277,15 +174476,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s15, s15, s16
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 9
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s16, v47, 8
-; SI-NEXT: v_readlane_b32 s17, v47, 4
+; SI-NEXT: s_add_i32 s15, s18, 3
+; SI-NEXT: v_readlane_b32 s16, v47, 6
+; SI-NEXT: v_readlane_b32 s17, v47, 2
; SI-NEXT: s_and_b32 s15, s15, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 8
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 5
+; SI-NEXT: v_readlane_b32 s16, v47, 3
; SI-NEXT: s_and_b32 s17, s17, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 24
; SI-NEXT: s_lshl_b32 s17, s17, 16
@@ -174293,15 +174491,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 7
+; SI-NEXT: v_readlane_b32 s16, v47, 5
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v47, 3
-; SI-NEXT: v_readlane_b32 s18, v56, 63
+; SI-NEXT: v_readlane_b32 s17, v47, 1
+; SI-NEXT: v_readlane_b32 s18, v56, 61
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v47, 0
+; SI-NEXT: v_readlane_b32 s17, v56, 62
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174310,15 +174508,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s40, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 61
+; SI-NEXT: v_readlane_b32 s16, v56, 59
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 60
-; SI-NEXT: v_readlane_b32 s18, v56, 57
+; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s18, v56, 55
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s17, v56, 56
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174327,15 +174525,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s41, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 59
+; SI-NEXT: v_readlane_b32 s16, v56, 57
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 56
-; SI-NEXT: v_readlane_b32 s18, v56, 53
+; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s18, v56, 51
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s17, v56, 52
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174344,15 +174542,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 51
+; SI-NEXT: v_readlane_b32 s16, v56, 49
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 50
-; SI-NEXT: v_readlane_b32 s18, v56, 46
+; SI-NEXT: v_readlane_b32 s17, v56, 48
+; SI-NEXT: v_readlane_b32 s18, v56, 44
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 47
+; SI-NEXT: v_readlane_b32 s17, v56, 45
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174361,15 +174559,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s43, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 49
+; SI-NEXT: v_readlane_b32 s16, v56, 47
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 48
-; SI-NEXT: v_readlane_b32 s18, v56, 44
+; SI-NEXT: v_readlane_b32 s17, v56, 46
+; SI-NEXT: v_readlane_b32 s18, v56, 42
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 45
+; SI-NEXT: v_readlane_b32 s17, v56, 43
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174378,15 +174576,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s44, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 43
+; SI-NEXT: v_readlane_b32 s16, v56, 41
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 42
-; SI-NEXT: v_readlane_b32 s18, v56, 38
+; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s18, v56, 36
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 39
+; SI-NEXT: v_readlane_b32 s17, v56, 37
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174395,14 +174593,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s45, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 41
+; SI-NEXT: v_readlane_b32 s16, v56, 39
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s17, v56, 38
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s87, 3
+; SI-NEXT: s_add_i32 s18, s85, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 37
+; SI-NEXT: v_readlane_b32 s17, v56, 35
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174414,7 +174612,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_add_i32 s16, s28, 3
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s25, 8
-; SI-NEXT: s_add_i32 s18, s85, 3
+; SI-NEXT: s_add_i32 s18, s30, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
@@ -174490,7 +174688,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_add_i32 s59, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s76, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 29
+; SI-NEXT: v_readlane_b32 s17, v56, 28
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s97, 3
@@ -174503,8 +174701,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s60, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 28
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s86, 3
; SI-NEXT: v_readlane_b32 s17, v56, 27
; SI-NEXT: v_readlane_b32 s18, v56, 26
; SI-NEXT: s_and_b32 s16, s16, 0xff
@@ -174521,7 +174718,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s61, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s77, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 34
+; SI-NEXT: v_readlane_b32 s17, v56, 32
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s99, 3
@@ -174534,15 +174731,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s62, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 33
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 32
-; SI-NEXT: v_readlane_b32 s18, v56, 31
+; SI-NEXT: s_add_i32 s16, s87, 3
+; SI-NEXT: v_readlane_b32 s17, v56, 31
+; SI-NEXT: v_readlane_b32 s18, v56, 30
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 30
+; SI-NEXT: v_readlane_b32 s17, v56, 29
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174570,12 +174766,12 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readlane_b32 s16, v56, 1
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: v_readlane_b32 s17, v56, 0
-; SI-NEXT: v_readlane_b32 s18, v56, 36
+; SI-NEXT: v_readlane_b32 s18, v56, 34
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 35
+; SI-NEXT: v_readlane_b32 s17, v56, 33
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174615,6 +174811,25 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s18, s18, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s75, s16, 0x3000000
+; SI-NEXT: s_lshr_b64 s[16:17], s[72:73], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 43
+; SI-NEXT: v_writelane_b32 v47, s17, 44
+; SI-NEXT: s_lshr_b64 s[16:17], s[62:63], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 45
+; SI-NEXT: v_writelane_b32 v47, s17, 46
+; SI-NEXT: s_lshr_b64 s[16:17], s[60:61], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 47
+; SI-NEXT: v_writelane_b32 v47, s17, 48
+; SI-NEXT: s_lshr_b64 s[16:17], s[58:59], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 49
+; SI-NEXT: v_writelane_b32 v47, s17, 50
+; SI-NEXT: s_lshr_b64 s[16:17], s[56:57], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 51
+; SI-NEXT: v_writelane_b32 v47, s17, 52
+; SI-NEXT: s_lshr_b64 s[16:17], s[46:47], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 53
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -174627,11 +174842,9 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_add_i32 s13, s13, 0x3000000
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s75, s16, 0x3000000
-; SI-NEXT: s_lshr_b64 s[30:31], s[72:73], 16
-; SI-NEXT: s_lshr_b64 s[94:95], s[62:63], 16
-; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 54
+; SI-NEXT: s_lshr_b64 s[16:17], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 55
; SI-NEXT: s_lshr_b64 s[38:39], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[50:51], s[10:11], 16
@@ -174639,18 +174852,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshr_b64 s[54:55], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[4:5], 16
; SI-NEXT: s_lshr_b64 s[82:83], s[74:75], 16
-; SI-NEXT: s_lshr_b64 vcc, s[60:61], 16
-; SI-NEXT: s_lshr_b64 s[70:71], s[58:59], 16
-; SI-NEXT: s_lshr_b64 s[80:81], s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[84:85], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[68:69], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 56
+; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[40:41], 16
; SI-NEXT: s_lshr_b32 s49, s75, 16
; SI-NEXT: s_lshr_b32 s51, s73, 16
; SI-NEXT: s_lshr_b32 s53, s63, 16
; SI-NEXT: s_lshr_b32 s55, s61, 16
; SI-NEXT: s_lshr_b32 s65, s59, 16
-; SI-NEXT: s_lshr_b32 s35, s57, 16
+; SI-NEXT: s_lshr_b32 s69, s57, 16
; SI-NEXT: s_lshr_b32 s79, s47, 16
; SI-NEXT: s_lshr_b32 s89, s45, 16
; SI-NEXT: s_lshr_b32 s91, s43, 16
@@ -174659,7 +174869,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: s_lshr_b32 s17, s11, 16
; SI-NEXT: s_lshr_b32 s92, s9, 16
-; SI-NEXT: s_lshr_b32 s78, s7, 16
+; SI-NEXT: s_lshr_b32 s68, s7, 16
; SI-NEXT: s_lshr_b32 s39, s5, 16
; SI-NEXT: .LBB93_5: ; %end
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -174672,45 +174882,57 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s16, s19
; SI-NEXT: s_and_b32 s19, s75, 0xffff
; SI-NEXT: s_lshl_b32 s20, s49, 16
+; SI-NEXT: v_readlane_b32 s22, v47, 43
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s20, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s30, 16
+; SI-NEXT: s_lshl_b32 s21, s22, 16
+; SI-NEXT: v_readlane_b32 s23, v47, 44
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_and_b32 s21, s73, 0xffff
; SI-NEXT: s_lshl_b32 s22, s51, 16
+; SI-NEXT: v_readlane_b32 s24, v47, 45
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s22, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s94, 16
+; SI-NEXT: s_lshl_b32 s23, s24, 16
+; SI-NEXT: v_readlane_b32 s25, v47, 46
; SI-NEXT: s_or_b32 s22, s22, s23
; SI-NEXT: s_and_b32 s23, s63, 0xffff
; SI-NEXT: s_lshl_b32 s24, s53, 16
+; SI-NEXT: v_readlane_b32 s26, v47, 47
; SI-NEXT: s_or_b32 s23, s23, s24
; SI-NEXT: s_and_b32 s24, s60, 0xffff
-; SI-NEXT: s_lshl_b32 s25, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s25, s26, 16
+; SI-NEXT: v_readlane_b32 s27, v47, 48
; SI-NEXT: s_or_b32 s24, s24, s25
; SI-NEXT: s_and_b32 s25, s61, 0xffff
; SI-NEXT: s_lshl_b32 s26, s55, 16
+; SI-NEXT: v_readlane_b32 s28, v47, 49
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s26, s58, 0xffff
-; SI-NEXT: s_lshl_b32 s27, s70, 16
+; SI-NEXT: s_lshl_b32 s27, s28, 16
+; SI-NEXT: v_readlane_b32 s29, v47, 50
; SI-NEXT: s_or_b32 s26, s26, s27
; SI-NEXT: s_and_b32 s27, s59, 0xffff
; SI-NEXT: s_lshl_b32 s28, s65, 16
+; SI-NEXT: v_readlane_b32 s58, v47, 51
; SI-NEXT: s_or_b32 s27, s27, s28
; SI-NEXT: s_and_b32 s28, s56, 0xffff
-; SI-NEXT: s_lshl_b32 s29, s80, 16
+; SI-NEXT: s_lshl_b32 s29, s58, 16
; SI-NEXT: s_or_b32 s28, s28, s29
; SI-NEXT: s_and_b32 s29, s57, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_lshl_b32 s56, s69, 16
; SI-NEXT: s_or_b32 s29, s29, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 53
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s84, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: v_readlane_b32 s57, v47, 54
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s47, s47, 0xffff
; SI-NEXT: s_lshl_b32 s56, s79, 16
; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 55
; SI-NEXT: s_and_b32 s44, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s68, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_or_b32 s44, s44, s56
; SI-NEXT: s_and_b32 s45, s45, 0xffff
; SI-NEXT: s_lshl_b32 s56, s89, 16
@@ -174748,7 +174970,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s56, s48, 16
; SI-NEXT: s_or_b32 s6, s6, s17
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s78, 16
+; SI-NEXT: s_lshl_b32 s17, s68, 16
; SI-NEXT: s_or_b32 s12, s12, s56
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s56, s31, 16
@@ -174764,6 +174986,8 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s10, s10, s56
; SI-NEXT: s_or_b32 s5, s5, s17
; SI-NEXT: v_readlane_b32 s30, v45, 34
+; SI-NEXT: v_readlane_b32 s59, v47, 52
+; SI-NEXT: v_readlane_b32 s57, v47, 56
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s19
; SI-NEXT: v_mov_b32_e32 v2, s20
@@ -194042,13 +194266,13 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s4, v21
; SI-NEXT: v_writelane_b32 v56, s4, 21
; SI-NEXT: v_readfirstlane_b32 s4, v20
+; SI-NEXT: v_writelane_b32 v56, s4, 22
+; SI-NEXT: v_readfirstlane_b32 s4, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:332
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:328
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:324
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:320
-; SI-NEXT: v_writelane_b32 v56, s4, 22
-; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:316
; SI-NEXT: v_writelane_b32 v56, s4, 23
; SI-NEXT: v_readfirstlane_b32 s4, v18
; SI-NEXT: v_writelane_b32 v56, s4, 24
@@ -194058,28 +194282,26 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_writelane_b32 v56, s4, 26
; SI-NEXT: v_readfirstlane_b32 s4, v15
; SI-NEXT: v_writelane_b32 v56, s4, 27
-; SI-NEXT: v_readfirstlane_b32 s4, v14
-; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v11
-; SI-NEXT: v_writelane_b32 v56, s4, 29
+; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v9
-; SI-NEXT: v_writelane_b32 v56, s4, 30
+; SI-NEXT: v_writelane_b32 v56, s4, 29
; SI-NEXT: v_readfirstlane_b32 s4, v8
-; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_writelane_b32 v56, s4, 30
; SI-NEXT: v_readfirstlane_b32 s4, v7
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:288
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:284
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:264
-; SI-NEXT: v_writelane_b32 v56, s4, 32
-; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_readfirstlane_b32 s87, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:280
; SI-NEXT: v_readfirstlane_b32 s98, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276
; SI-NEXT: v_readfirstlane_b32 s99, v4
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272
-; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268
+; SI-NEXT: v_readfirstlane_b32 s86, v14
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:312
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:308
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292
@@ -194089,34 +194311,35 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
; SI-NEXT: v_readfirstlane_b32 s76, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:296
-; SI-NEXT: v_writelane_b32 v56, s4, 34
+; SI-NEXT: v_writelane_b32 v56, s4, 32
; SI-NEXT: v_readfirstlane_b32 s77, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260
; SI-NEXT: ; implicit-def: $vgpr47 : SGPR spill to VGPR lane
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v22
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 s81, v21
+; SI-NEXT: v_readfirstlane_b32 s83, v21
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:228
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:224
; SI-NEXT: v_readfirstlane_b32 s4, v1
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:252
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:248
-; SI-NEXT: v_writelane_b32 v56, s4, 35
+; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v0
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:244
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
-; SI-NEXT: v_readfirstlane_b32 s82, v23
+; SI-NEXT: v_readfirstlane_b32 s90, v23
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:220
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:216
-; SI-NEXT: v_readfirstlane_b32 s80, v20
+; SI-NEXT: v_readfirstlane_b32 s82, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:232
-; SI-NEXT: v_readfirstlane_b32 s83, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:232
+; SI-NEXT: v_readfirstlane_b32 s94, v20
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:208
-; SI-NEXT: v_writelane_b32 v56, s4, 36
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:208
+; SI-NEXT: v_writelane_b32 v47, s94, 0
+; SI-NEXT: v_writelane_b32 v56, s4, 34
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s21, v8
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:156
@@ -194127,56 +194350,56 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s46, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:148
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:144
-; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_readfirstlane_b32 s63, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140
; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:136
-; SI-NEXT: v_readfirstlane_b32 s73, v5
-; SI-NEXT: v_readfirstlane_b32 s40, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:128
+; SI-NEXT: v_readfirstlane_b32 s40, v4
; SI-NEXT: v_readfirstlane_b32 s69, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
; SI-NEXT: v_readfirstlane_b32 s47, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
; SI-NEXT: v_readfirstlane_b32 s55, v9
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:164
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
-; SI-NEXT: v_readfirstlane_b32 s57, v14
+; SI-NEXT: v_readfirstlane_b32 s56, v14
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:192
; SI-NEXT: v_readfirstlane_b32 s16, v12
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
; SI-NEXT: v_readfirstlane_b32 s54, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:172
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
-; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:180
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:176
-; SI-NEXT: v_readfirstlane_b32 s60, v24
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v21
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 s59, v24
+; SI-NEXT: v_readfirstlane_b32 s62, v21
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:28
; SI-NEXT: s_waitcnt expcnt(3)
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
-; SI-NEXT: v_readfirstlane_b32 s84, v17
+; SI-NEXT: v_readfirstlane_b32 s74, v17
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
-; SI-NEXT: v_readfirstlane_b32 s90, v18
+; SI-NEXT: v_readfirstlane_b32 s29, v18
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
-; SI-NEXT: v_readfirstlane_b32 s23, v25
+; SI-NEXT: v_readfirstlane_b32 s72, v25
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:20
; SI-NEXT: s_waitcnt expcnt(2)
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:16
-; SI-NEXT: v_readfirstlane_b32 s65, v23
+; SI-NEXT: v_readfirstlane_b32 s64, v23
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:12
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:8
@@ -194186,190 +194409,187 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s37, v22
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:52
; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
-; SI-NEXT: v_readfirstlane_b32 s56, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:44
+; SI-NEXT: v_readfirstlane_b32 s71, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
-; SI-NEXT: v_readfirstlane_b32 s17, v16
+; SI-NEXT: v_readfirstlane_b32 s58, v16
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
-; SI-NEXT: v_readfirstlane_b32 s62, v1
+; SI-NEXT: v_readfirstlane_b32 s60, v1
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:80
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:36
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
-; SI-NEXT: v_readfirstlane_b32 s94, v29
+; SI-NEXT: v_readfirstlane_b32 s65, v29
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32
-; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s13, v11
-; SI-NEXT: v_readfirstlane_b32 s66, v33
+; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_readfirstlane_b32 s66, v33
; SI-NEXT: v_readfirstlane_b32 s14, v34
; SI-NEXT: v_readfirstlane_b32 s15, v6
; SI-NEXT: v_readfirstlane_b32 s93, v39
-; SI-NEXT: v_readfirstlane_b32 s39, v35
-; SI-NEXT: v_readfirstlane_b32 s49, v4
-; SI-NEXT: v_readfirstlane_b32 s51, v37
-; SI-NEXT: v_writelane_b32 v47, s51, 0
-; SI-NEXT: v_writelane_b32 v47, s54, 1
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s12, v38
-; SI-NEXT: v_writelane_b32 v47, s83, 2
+; SI-NEXT: v_writelane_b32 v47, s12, 1
+; SI-NEXT: v_readfirstlane_b32 s57, v26
; SI-NEXT: v_readfirstlane_b32 s52, v9
-; SI-NEXT: v_writelane_b32 v47, s12, 3
-; SI-NEXT: v_writelane_b32 v47, s52, 4
+; SI-NEXT: v_writelane_b32 v47, s52, 2
; SI-NEXT: v_readfirstlane_b32 s67, v32
-; SI-NEXT: v_readfirstlane_b32 s58, v14
-; SI-NEXT: v_readfirstlane_b32 s59, v26
-; SI-NEXT: v_readfirstlane_b32 s61, v27
+; SI-NEXT: v_readfirstlane_b32 s80, v14
+; SI-NEXT: v_readfirstlane_b32 s17, v27
+; SI-NEXT: v_readfirstlane_b32 s61, v12
+; SI-NEXT: v_readfirstlane_b32 s73, v28
; SI-NEXT: v_readfirstlane_b32 s53, v31
-; SI-NEXT: v_writelane_b32 v47, s53, 5
-; SI-NEXT: v_writelane_b32 v47, s55, 6
-; SI-NEXT: v_writelane_b32 v47, s13, 7
-; SI-NEXT: v_writelane_b32 v47, s67, 8
-; SI-NEXT: v_writelane_b32 v47, s88, 9
-; SI-NEXT: v_writelane_b32 v47, s44, 10
-; SI-NEXT: v_writelane_b32 v47, s45, 11
-; SI-NEXT: v_writelane_b32 v47, s46, 12
-; SI-NEXT: v_writelane_b32 v47, s78, 13
-; SI-NEXT: v_writelane_b32 v47, s66, 14
-; SI-NEXT: v_writelane_b32 v47, s47, 15
-; SI-NEXT: v_writelane_b32 v47, s56, 16
-; SI-NEXT: v_writelane_b32 v47, s57, 17
-; SI-NEXT: v_writelane_b32 v47, s14, 18
-; SI-NEXT: v_writelane_b32 v47, s58, 19
-; SI-NEXT: v_writelane_b32 v47, s59, 20
-; SI-NEXT: v_writelane_b32 v47, s17, 21
-; SI-NEXT: v_writelane_b32 v47, s60, 22
-; SI-NEXT: v_writelane_b32 v47, s15, 23
-; SI-NEXT: v_writelane_b32 v47, s61, 24
-; SI-NEXT: v_readfirstlane_b32 s63, v12
-; SI-NEXT: v_writelane_b32 v47, s62, 25
-; SI-NEXT: v_writelane_b32 v47, s63, 26
-; SI-NEXT: v_writelane_b32 v47, s16, 27
+; SI-NEXT: v_writelane_b32 v47, s53, 3
+; SI-NEXT: v_writelane_b32 v47, s55, 4
+; SI-NEXT: v_writelane_b32 v47, s13, 5
+; SI-NEXT: v_writelane_b32 v47, s67, 6
+; SI-NEXT: v_writelane_b32 v47, s88, 7
+; SI-NEXT: v_writelane_b32 v47, s44, 8
+; SI-NEXT: v_writelane_b32 v47, s45, 9
+; SI-NEXT: v_writelane_b32 v47, s46, 10
+; SI-NEXT: v_writelane_b32 v47, s78, 11
+; SI-NEXT: v_writelane_b32 v47, s66, 12
+; SI-NEXT: v_writelane_b32 v47, s71, 13
+; SI-NEXT: v_writelane_b32 v47, s14, 14
+; SI-NEXT: v_writelane_b32 v47, s80, 15
+; SI-NEXT: v_writelane_b32 v47, s57, 16
+; SI-NEXT: v_writelane_b32 v47, s58, 17
+; SI-NEXT: v_writelane_b32 v47, s59, 18
+; SI-NEXT: v_writelane_b32 v47, s15, 19
+; SI-NEXT: v_writelane_b32 v47, s17, 20
+; SI-NEXT: v_writelane_b32 v47, s60, 21
; SI-NEXT: v_readfirstlane_b32 s92, v2
-; SI-NEXT: v_writelane_b32 v47, vcc_lo, 28
-; SI-NEXT: v_writelane_b32 v47, s72, 29
-; SI-NEXT: v_writelane_b32 v47, s73, 30
-; SI-NEXT: v_writelane_b32 v47, s23, 31
-; SI-NEXT: v_readfirstlane_b32 s74, v13
+; SI-NEXT: v_writelane_b32 v47, s61, 22
+; SI-NEXT: v_writelane_b32 v47, s16, 23
+; SI-NEXT: v_writelane_b32 v47, s62, 24
+; SI-NEXT: v_writelane_b32 v47, s63, 25
; SI-NEXT: v_readfirstlane_b32 s95, v48
-; SI-NEXT: v_writelane_b32 v47, s18, 32
-; SI-NEXT: v_readfirstlane_b32 s64, v28
-; SI-NEXT: v_writelane_b32 v47, s74, 33
-; SI-NEXT: v_writelane_b32 v47, s64, 34
+; SI-NEXT: v_writelane_b32 v47, vcc_lo, 26
+; SI-NEXT: v_writelane_b32 v47, s72, 27
+; SI-NEXT: v_readfirstlane_b32 s23, v13
+; SI-NEXT: v_writelane_b32 v47, s18, 28
+; SI-NEXT: v_writelane_b32 v47, s23, 29
+; SI-NEXT: v_writelane_b32 v47, s73, 30
; SI-NEXT: v_readfirstlane_b32 s41, v30
+; SI-NEXT: v_writelane_b32 v47, s40, 31
+; SI-NEXT: v_writelane_b32 v47, s41, 32
+; SI-NEXT: v_readfirstlane_b32 s68, v10
+; SI-NEXT: v_readfirstlane_b32 s38, v5
+; SI-NEXT: v_writelane_b32 v47, s74, 33
+; SI-NEXT: v_readfirstlane_b32 s39, v35
+; SI-NEXT: v_writelane_b32 v47, s68, 34
+; SI-NEXT: v_writelane_b32 v47, s64, 35
; SI-NEXT: v_readfirstlane_b32 s19, v41
-; SI-NEXT: v_writelane_b32 v56, s19, 37
+; SI-NEXT: v_writelane_b32 v56, s19, 35
; SI-NEXT: v_readfirstlane_b32 s43, v51
; SI-NEXT: v_readfirstlane_b32 s42, v18
-; SI-NEXT: v_writelane_b32 v56, s42, 38
-; SI-NEXT: v_writelane_b32 v56, s43, 39
+; SI-NEXT: v_writelane_b32 v56, s42, 36
+; SI-NEXT: v_writelane_b32 v56, s43, 37
; SI-NEXT: v_readfirstlane_b32 s6, v42
; SI-NEXT: v_readfirstlane_b32 s7, v23
-; SI-NEXT: v_writelane_b32 v56, s6, 40
+; SI-NEXT: v_writelane_b32 v56, s6, 38
; SI-NEXT: v_readfirstlane_b32 s75, v52
-; SI-NEXT: v_writelane_b32 v56, s7, 41
+; SI-NEXT: v_writelane_b32 v56, s7, 39
; SI-NEXT: s_waitcnt vmcnt(13)
; SI-NEXT: v_readfirstlane_b32 s79, v0
-; SI-NEXT: v_writelane_b32 v56, s75, 42
+; SI-NEXT: v_writelane_b32 v56, s75, 40
; SI-NEXT: s_waitcnt vmcnt(11)
; SI-NEXT: v_readfirstlane_b32 s89, v22
-; SI-NEXT: v_writelane_b32 v56, s79, 43
+; SI-NEXT: v_writelane_b32 v56, s79, 41
; SI-NEXT: v_readfirstlane_b32 s91, v53
-; SI-NEXT: v_writelane_b32 v56, s89, 44
-; SI-NEXT: v_writelane_b32 v56, s91, 45
-; SI-NEXT: v_writelane_b32 v56, s92, 46
+; SI-NEXT: v_writelane_b32 v56, s89, 42
+; SI-NEXT: v_writelane_b32 v56, s91, 43
+; SI-NEXT: v_writelane_b32 v56, s92, 44
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_readfirstlane_b32 s8, v54
-; SI-NEXT: v_writelane_b32 v56, s93, 47
+; SI-NEXT: v_writelane_b32 v56, s93, 45
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s9, v20
-; SI-NEXT: v_writelane_b32 v56, s8, 48
-; SI-NEXT: v_writelane_b32 v56, s9, 49
+; SI-NEXT: v_readfirstlane_b32 s9, v19
+; SI-NEXT: v_writelane_b32 v56, s8, 46
+; SI-NEXT: v_writelane_b32 v56, s9, 47
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s31, v16
-; SI-NEXT: v_writelane_b32 v56, s95, 50
-; SI-NEXT: v_writelane_b32 v56, s31, 51
+; SI-NEXT: v_writelane_b32 v56, s95, 48
+; SI-NEXT: v_writelane_b32 v56, s31, 49
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_writelane_b32 v56, s80, 52
-; SI-NEXT: v_writelane_b32 v47, s40, 35
+; SI-NEXT: v_writelane_b32 v56, s82, 50
; SI-NEXT: v_readfirstlane_b32 s36, v49
-; SI-NEXT: v_writelane_b32 v56, s34, 53
-; SI-NEXT: v_writelane_b32 v47, s41, 36
-; SI-NEXT: v_readfirstlane_b32 s68, v10
-; SI-NEXT: v_writelane_b32 v56, s36, 54
-; SI-NEXT: v_writelane_b32 v47, s84, 37
+; SI-NEXT: v_writelane_b32 v56, s34, 51
+; SI-NEXT: v_writelane_b32 v56, s36, 52
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_readfirstlane_b32 s10, v50
-; SI-NEXT: v_writelane_b32 v56, s81, 55
-; SI-NEXT: v_writelane_b32 v47, s68, 38
-; SI-NEXT: v_readfirstlane_b32 s38, v5
-; SI-NEXT: v_writelane_b32 v56, s10, 56
-; SI-NEXT: v_writelane_b32 v47, s65, 39
-; SI-NEXT: v_writelane_b32 v56, s38, 57
-; SI-NEXT: v_writelane_b32 v47, s90, 40
+; SI-NEXT: v_writelane_b32 v56, s83, 53
+; SI-NEXT: v_writelane_b32 v56, s10, 54
+; SI-NEXT: v_writelane_b32 v56, s38, 55
; SI-NEXT: v_readfirstlane_b32 s11, v17
-; SI-NEXT: v_writelane_b32 v56, s39, 58
-; SI-NEXT: v_writelane_b32 v47, s69, 41
+; SI-NEXT: v_writelane_b32 v56, s39, 56
; SI-NEXT: v_readfirstlane_b32 s48, v36
-; SI-NEXT: v_writelane_b32 v56, s11, 59
-; SI-NEXT: v_writelane_b32 v47, s94, 42
-; SI-NEXT: v_writelane_b32 v56, s48, 60
-; SI-NEXT: v_writelane_b32 v47, s70, 43
-; SI-NEXT: v_readfirstlane_b32 s35, v19
-; SI-NEXT: v_writelane_b32 v56, s49, 61
-; SI-NEXT: v_writelane_b32 v47, s37, 44
-; SI-NEXT: v_readfirstlane_b32 s71, v15
+; SI-NEXT: v_writelane_b32 v56, s11, 57
+; SI-NEXT: v_writelane_b32 v47, s29, 36
+; SI-NEXT: v_readfirstlane_b32 s49, v4
+; SI-NEXT: v_writelane_b32 v56, s48, 58
+; SI-NEXT: v_writelane_b32 v47, s69, 37
+; SI-NEXT: v_writelane_b32 v56, s49, 59
+; SI-NEXT: v_writelane_b32 v47, s65, 38
; SI-NEXT: v_readfirstlane_b32 s50, v3
+; SI-NEXT: v_writelane_b32 v56, s90, 60
+; SI-NEXT: v_writelane_b32 v47, s70, 39
+; SI-NEXT: v_readfirstlane_b32 s35, v20
+; SI-NEXT: v_readfirstlane_b32 s51, v37
+; SI-NEXT: v_writelane_b32 v56, s50, 61
+; SI-NEXT: v_writelane_b32 v47, s37, 40
+; SI-NEXT: v_readfirstlane_b32 s81, v15
; SI-NEXT: v_readfirstlane_b32 s22, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s85, v24
+; SI-NEXT: v_readfirstlane_b32 s30, v24
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s25, v40
; SI-NEXT: v_readfirstlane_b32 s28, v21
-; SI-NEXT: v_readfirstlane_b32 s87, v25
+; SI-NEXT: v_readfirstlane_b32 s85, v25
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v43
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_readfirstlane_b32 s26, v29
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s27, v44
-; SI-NEXT: v_writelane_b32 v56, s82, 62
-; SI-NEXT: v_writelane_b32 v47, s35, 45
-; SI-NEXT: v_writelane_b32 v56, s50, 63
-; SI-NEXT: v_writelane_b32 v47, s71, 46
+; SI-NEXT: v_writelane_b32 v56, s51, 62
+; SI-NEXT: v_writelane_b32 v47, s35, 41
+; SI-NEXT: v_writelane_b32 v56, s54, 63
+; SI-NEXT: v_writelane_b32 v47, s81, 42
; SI-NEXT: s_cbranch_scc0 .LBB97_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: v_readlane_b32 s4, v56, 11
; SI-NEXT: v_readlane_b32 s5, v56, 10
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: v_writelane_b32 v47, s21, 47
+; SI-NEXT: v_writelane_b32 v47, s21, 57
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 4
-; SI-NEXT: v_writelane_b32 v47, s4, 48
+; SI-NEXT: v_writelane_b32 v47, s4, 43
; SI-NEXT: s_and_b32 s4, s24, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 34
-; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_readlane_b32 s5, v56, 32
+; SI-NEXT: v_writelane_b32 v47, s4, 45
; SI-NEXT: s_and_b32 s4, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 29
-; SI-NEXT: v_writelane_b32 v47, s4, 50
+; SI-NEXT: v_readlane_b32 s5, v56, 28
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s4, 49
; SI-NEXT: v_readlane_b32 s4, v56, 24
; SI-NEXT: s_and_b32 s5, s4, 0xff
; SI-NEXT: v_readlane_b32 s4, v56, 23
; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_writelane_b32 v47, s4, 52
+; SI-NEXT: v_writelane_b32 v47, s4, 51
; SI-NEXT: v_readlane_b32 s4, v56, 16
; SI-NEXT: v_readlane_b32 s5, v56, 15
; SI-NEXT: s_and_b32 s4, s4, 0xff
@@ -194379,41 +194599,45 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s4, s7, 0xff
; SI-NEXT: s_lshl_b32 s5, s6, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 54
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_and_b32 s4, s9, 0xff
; SI-NEXT: s_lshl_b32 s5, s8, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s4, 58
; SI-NEXT: s_and_b32 s4, s11, 0xff
; SI-NEXT: s_lshl_b32 s5, s10, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 56
+; SI-NEXT: v_writelane_b32 v47, s4, 59
; SI-NEXT: s_and_b32 s4, s13, 0xff
; SI-NEXT: s_lshl_b32 s5, s12, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 57
+; SI-NEXT: v_writelane_b32 v47, s4, 60
; SI-NEXT: s_and_b32 s4, s15, 0xff
; SI-NEXT: s_lshl_b32 s5, s14, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 58
+; SI-NEXT: v_writelane_b32 v47, s4, 61
; SI-NEXT: s_and_b32 s4, s68, 0xff
; SI-NEXT: s_lshl_b32 s5, s41, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 59
-; SI-NEXT: s_and_b32 s4, s71, 0xff
+; SI-NEXT: v_writelane_b32 v47, s4, 62
+; SI-NEXT: s_and_b32 s4, s81, 0xff
; SI-NEXT: s_lshl_b32 s5, s35, 8
-; SI-NEXT: s_or_b32 s71, s4, s5
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v47, s4, 63
; SI-NEXT: s_and_b32 s4, s37, 0xff
; SI-NEXT: s_lshl_b32 s5, s70, 8
-; SI-NEXT: s_or_b32 s70, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_lshl_b32 s5, s40, 8
-; SI-NEXT: s_or_b32 s69, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 0
+; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 1
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s16, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 7
-; SI-NEXT: v_writelane_b32 v47, s4, 60
+; SI-NEXT: v_writelane_b32 v46, s4, 2
; SI-NEXT: s_and_b32 s4, s20, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s5, s4, s5
@@ -194422,57 +194646,52 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readlane_b32 s6, v56, 5
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_or_b32 s7, s6, s4
+; SI-NEXT: s_or_b32 s81, s6, s4
; SI-NEXT: v_readlane_b32 s4, v56, 9
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: v_readlane_b32 s6, v56, 8
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s7, 61
; SI-NEXT: s_or_b32 s4, s6, s4
-; SI-NEXT: s_or_b32 s5, s5, s7
; SI-NEXT: v_readlane_b32 s6, v56, 1
; SI-NEXT: v_readlane_b32 s7, v56, 0
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_or_b32 s7, s6, s7
-; SI-NEXT: v_readlane_b32 s6, v56, 36
+; SI-NEXT: v_readlane_b32 s6, v56, 34
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 35
+; SI-NEXT: v_readlane_b32 s8, v56, 33
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_or_b32 s9, s8, s6
; SI-NEXT: v_readlane_b32 s6, v56, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: v_readlane_b32 s8, v56, 2
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s9, 3
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s9, 62
-; SI-NEXT: s_or_b32 s6, s8, s6
; SI-NEXT: s_or_b32 s7, s7, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 33
-; SI-NEXT: v_readlane_b32 s9, v56, 32
-; SI-NEXT: s_and_b32 s8, s8, 0xff
+; SI-NEXT: v_readlane_b32 s9, v56, 31
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_and_b32 s8, s87, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
; SI-NEXT: s_or_b32 s9, s8, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 31
+; SI-NEXT: v_readlane_b32 s8, v56, 30
; SI-NEXT: s_and_b32 s8, s8, 0xff
-; SI-NEXT: v_readlane_b32 s10, v56, 30
+; SI-NEXT: v_readlane_b32 s10, v56, 29
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s10, 24
; SI-NEXT: s_or_b32 s11, s10, s8
; SI-NEXT: s_and_b32 s8, s99, 0xff
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s11, 4
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s98, 24
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s11, 63
-; SI-NEXT: s_or_b32 s8, s10, s8
; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: v_readlane_b32 s10, v56, 28
; SI-NEXT: v_readlane_b32 s11, v56, 27
-; SI-NEXT: s_and_b32 s10, s10, 0xff
+; SI-NEXT: s_or_b32 s8, s10, s8
+; SI-NEXT: s_and_b32 s10, s86, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 8
; SI-NEXT: s_or_b32 s11, s10, s11
; SI-NEXT: v_readlane_b32 s10, v56, 26
@@ -194480,16 +194699,12 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readlane_b32 s12, v56, 25
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s13, s12, s10
+; SI-NEXT: s_or_b32 s84, s12, s10
; SI-NEXT: s_and_b32 s10, s97, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s96, 24
-; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: v_readlane_b32 s12, v56, 20
-; SI-NEXT: v_writelane_b32 v46, s13, 0
-; SI-NEXT: s_or_b32 s11, s11, s13
; SI-NEXT: s_and_b32 s13, s12, 0xff
; SI-NEXT: v_readlane_b32 s12, v56, 19
; SI-NEXT: s_lshl_b32 s12, s12, 8
@@ -194499,7 +194714,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s14, s12, 16
; SI-NEXT: v_readlane_b32 s12, v56, 17
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s86, s12, s14
+; SI-NEXT: s_or_b32 s69, s12, s14
; SI-NEXT: v_readlane_b32 s12, v56, 22
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s14, s12, 16
@@ -194513,7 +194728,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s14, s26, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_lshl_b32 s16, vcc_hi, 24
-; SI-NEXT: s_or_b32 s29, s16, s14
+; SI-NEXT: s_or_b32 s70, s16, s14
; SI-NEXT: v_readlane_b32 s14, v56, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s16, s14, 16
@@ -194523,16 +194738,16 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s16, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s25, 8
; SI-NEXT: s_or_b32 s18, s16, s18
-; SI-NEXT: s_and_b32 s16, s85, 0xff
-; SI-NEXT: v_writelane_b32 v46, s20, 1
+; SI-NEXT: s_and_b32 s16, s30, 0xff
+; SI-NEXT: v_writelane_b32 v46, s20, 5
; SI-NEXT: s_lshl_b32 s16, s16, 16
; SI-NEXT: s_lshl_b32 s20, s22, 24
-; SI-NEXT: v_writelane_b32 v46, vcc_hi, 2
-; SI-NEXT: v_writelane_b32 v46, s28, 3
+; SI-NEXT: v_writelane_b32 v46, vcc_hi, 6
+; SI-NEXT: v_writelane_b32 v46, s28, 7
; SI-NEXT: s_or_b32 s28, s20, s16
-; SI-NEXT: s_and_b32 s20, s87, 0xff
+; SI-NEXT: s_and_b32 s20, s85, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: v_writelane_b32 v46, s24, 4
+; SI-NEXT: v_writelane_b32 v46, s24, 8
; SI-NEXT: s_lshl_b32 s24, s19, 24
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s40, s24, s20
@@ -194546,7 +194761,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s19, s24, s18
; SI-NEXT: s_and_b32 s24, s89, 0xff
; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: v_writelane_b32 v46, s25, 5
+; SI-NEXT: v_writelane_b32 v46, s25, 9
; SI-NEXT: s_lshl_b32 s25, s91, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s42, s25, s24
@@ -194571,7 +194786,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s25, s38, 0xff
; SI-NEXT: s_lshl_b32 s25, s25, 16
; SI-NEXT: s_lshl_b32 s26, s39, 24
-; SI-NEXT: v_writelane_b32 v46, s27, 6
+; SI-NEXT: v_writelane_b32 v46, s27, 10
; SI-NEXT: s_or_b32 s27, s26, s25
; SI-NEXT: s_and_b32 s26, s50, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
@@ -194592,44 +194807,44 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s38, s75, s26
; SI-NEXT: s_or_b32 s39, s20, s18
-; SI-NEXT: s_and_b32 s20, s63, 0xff
-; SI-NEXT: s_lshl_b32 s26, s61, 8
+; SI-NEXT: s_and_b32 s20, s61, 0xff
+; SI-NEXT: s_lshl_b32 s26, s17, 8
; SI-NEXT: s_or_b32 s20, s20, s26
-; SI-NEXT: s_and_b32 s26, s58, 0xff
+; SI-NEXT: s_and_b32 s26, s80, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
-; SI-NEXT: s_lshl_b32 s75, s59, 24
+; SI-NEXT: s_lshl_b32 s75, s57, 24
; SI-NEXT: s_or_b32 s26, s75, s26
-; SI-NEXT: s_and_b32 s75, s74, 0xff
+; SI-NEXT: s_and_b32 s75, s23, 0xff
; SI-NEXT: s_lshl_b32 s75, s75, 16
-; SI-NEXT: s_lshl_b32 s78, s64, 24
+; SI-NEXT: s_lshl_b32 s78, s73, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s48, s78, s75
; SI-NEXT: s_or_b32 s49, s20, s26
-; SI-NEXT: s_and_b32 s20, s23, 0xff
-; SI-NEXT: s_lshl_b32 s75, vcc_lo, 8
+; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_lshl_b32 s75, s62, 8
; SI-NEXT: s_or_b32 s75, s20, s75
-; SI-NEXT: s_and_b32 s20, s60, 0xff
+; SI-NEXT: s_and_b32 s20, s59, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_lshl_b32 s78, s56, 24
+; SI-NEXT: s_lshl_b32 s78, s71, 24
; SI-NEXT: s_or_b32 s20, s78, s20
-; SI-NEXT: s_and_b32 s78, s94, 0xff
+; SI-NEXT: s_and_b32 s78, s65, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
-; SI-NEXT: s_lshl_b32 s79, s65, 24
+; SI-NEXT: s_lshl_b32 s79, s64, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_or_b32 s50, s79, s78
; SI-NEXT: s_or_b32 s51, s75, s20
-; SI-NEXT: s_and_b32 s75, s62, 0xff
-; SI-NEXT: s_lshl_b32 s78, s17, 8
+; SI-NEXT: s_and_b32 s75, s60, 0xff
+; SI-NEXT: s_lshl_b32 s78, s58, 8
; SI-NEXT: s_or_b32 s75, s75, s78
; SI-NEXT: s_and_b32 s78, s45, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
; SI-NEXT: s_lshl_b32 s79, s44, 24
; SI-NEXT: s_or_b32 s78, s79, s78
-; SI-NEXT: s_and_b32 s79, s90, 0xff
+; SI-NEXT: s_and_b32 s79, s29, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s88, s84, 24
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 47
+; SI-NEXT: v_readlane_b32 s16, v47, 57
; SI-NEXT: s_or_b32 s52, s88, s79
; SI-NEXT: s_or_b32 s53, s75, s78
; SI-NEXT: s_and_b32 s75, s46, 0xff
@@ -194639,167 +194854,188 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_lshl_b32 s88, s54, 24
; SI-NEXT: s_or_b32 s88, s88, s79
-; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_and_b32 s79, vcc_lo, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s72, 24
+; SI-NEXT: s_lshl_b32 s89, s63, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 48
+; SI-NEXT: v_readlane_b32 s16, v47, 43
; SI-NEXT: s_or_b32 s54, s89, s79
; SI-NEXT: s_or_b32 s55, s75, s88
-; SI-NEXT: s_and_b32 s75, s83, 0xff
-; SI-NEXT: s_lshl_b32 s79, s82, 8
+; SI-NEXT: s_and_b32 s75, s94, 0xff
+; SI-NEXT: s_lshl_b32 s79, s90, 8
; SI-NEXT: s_and_b32 s44, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 49
+; SI-NEXT: v_readlane_b32 s16, v47, 45
; SI-NEXT: s_or_b32 s75, s75, s79
-; SI-NEXT: s_and_b32 s79, s81, 0xff
+; SI-NEXT: s_and_b32 s79, s83, 0xff
; SI-NEXT: s_and_b32 s45, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 50
+; SI-NEXT: v_readlane_b32 s16, v47, 47
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s80, 24
+; SI-NEXT: s_lshl_b32 s89, s82, 24
; SI-NEXT: s_and_b32 s46, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: v_readlane_b32 s16, v47, 49
; SI-NEXT: s_or_b32 s90, s89, s79
+; SI-NEXT: s_mov_b32 s71, s47
; SI-NEXT: s_lshl_b32 s89, s47, 24
; SI-NEXT: s_and_b32 s47, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 52
-; SI-NEXT: s_and_b32 s79, s57, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: s_and_b32 s79, s56, 0xff
; SI-NEXT: s_and_b32 s56, s16, 0xffff
; SI-NEXT: v_readlane_b32 s16, v47, 53
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_and_b32 s57, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 54
+; SI-NEXT: v_readlane_b32 s16, v47, 55
; SI-NEXT: s_or_b32 s64, s89, s79
; SI-NEXT: s_and_b32 s79, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 55
+; SI-NEXT: v_readlane_b32 s16, v47, 58
; SI-NEXT: s_and_b32 s89, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 56
+; SI-NEXT: v_readlane_b32 s16, v47, 59
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_and_b32 s17, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 57
+; SI-NEXT: v_readlane_b32 s16, v47, 60
+; SI-NEXT: s_or_b32 s5, s5, s81
+; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_and_b32 s91, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 58
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: s_or_b32 s65, s75, s90
; SI-NEXT: s_and_b32 s92, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 59
-; SI-NEXT: s_or_b32 s13, s13, s86
+; SI-NEXT: v_readlane_b32 s16, v47, 62
+; SI-NEXT: s_or_b32 s74, s44, s4
+; SI-NEXT: s_mov_b32 s75, s5
+; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[4:5], s[6:7], 16
; SI-NEXT: s_and_b32 s93, s16, 0xffff
-; SI-NEXT: s_mov_b32 s16, s85
-; SI-NEXT: s_lshr_b64 s[84:85], s[40:41], 16
-; SI-NEXT: s_and_b32 s94, s71, 0xffff
-; SI-NEXT: s_and_b32 s95, s70, 0xffff
-; SI-NEXT: s_or_b32 s58, s56, s12
-; SI-NEXT: s_mov_b32 s59, s13
-; SI-NEXT: s_lshr_b64 s[70:71], s[12:13], 16
-; SI-NEXT: s_or_b32 s12, s93, s48
-; SI-NEXT: s_mov_b32 s13, s49
-; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
-; SI-NEXT: s_mov_b32 s85, s16
-; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: v_readlane_b32 s16, v47, 63
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 16
+; SI-NEXT: s_or_b32 s11, s11, s84
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: s_lshr_b64 s[4:5], s[10:11], 16
+; SI-NEXT: s_or_b32 s13, s13, s69
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_mov_b32 s25, s22
-; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s22, v47, 60
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: s_lshr_b64 s[4:5], s[12:13], 16
+; SI-NEXT: s_or_b32 s15, s15, s70
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: s_lshr_b64 s[4:5], s[14:15], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: s_lshr_b64 s[4:5], s[40:41], 16
+; SI-NEXT: s_mov_b32 s23, s21
+; SI-NEXT: v_readlane_b32 s21, v46, 1
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: s_and_b32 s94, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_and_b32 vcc_lo, s21, 0xffff
+; SI-NEXT: v_readlane_b32 s21, v46, 2
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: s_lshr_b64 s[4:5], s[42:43], 16
+; SI-NEXT: s_and_b32 s95, s16, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s21, 0xffff
; SI-NEXT: s_or_b32 s60, s47, s10
; SI-NEXT: s_mov_b32 s61, s11
-; SI-NEXT: s_lshr_b64 vcc, s[10:11], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_or_b32 s10, s94, s50
; SI-NEXT: s_mov_b32 s11, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[50:51], 16
-; SI-NEXT: s_lshr_b32 s49, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 62
-; SI-NEXT: s_or_b32 s15, s15, s29
-; SI-NEXT: s_mov_b32 s23, s21
-; SI-NEXT: s_or_b32 s65, s75, s90
-; SI-NEXT: s_and_b32 s21, s69, 0xffff
-; SI-NEXT: s_and_b32 s22, s22, 0xffff
-; SI-NEXT: s_or_b32 s62, s46, s8
-; SI-NEXT: s_mov_b32 s63, s9
-; SI-NEXT: s_lshr_b64 s[66:67], s[8:9], 16
-; SI-NEXT: s_or_b32 s8, s95, s52
-; SI-NEXT: s_mov_b32 s9, s53
-; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
-; SI-NEXT: s_lshr_b32 s51, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 63
-; SI-NEXT: s_or_b32 s74, s44, s4
-; SI-NEXT: s_mov_b32 s75, s5
-; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: v_readlane_b32 s16, v46, 3
; SI-NEXT: s_or_b32 s72, s45, s6
; SI-NEXT: s_mov_b32 s73, s7
-; SI-NEXT: s_lshr_b64 s[30:31], s[6:7], 16
+; SI-NEXT: s_or_b32 s62, s46, s8
+; SI-NEXT: s_mov_b32 s63, s9
+; SI-NEXT: s_or_b32 s58, s56, s12
+; SI-NEXT: s_mov_b32 s59, s13
; SI-NEXT: s_or_b32 s56, s57, s14
; SI-NEXT: s_mov_b32 s57, s15
-; SI-NEXT: s_lshr_b64 s[80:81], s[14:15], 16
-; SI-NEXT: s_or_b32 s44, s89, s42
-; SI-NEXT: s_mov_b32 s45, s43
-; SI-NEXT: s_lshr_b64 s[68:69], s[42:43], 16
-; SI-NEXT: s_or_b32 s42, s17, s34
-; SI-NEXT: s_mov_b32 s43, s35
-; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
+; SI-NEXT: v_writelane_b32 v47, s5, 56
; SI-NEXT: s_or_b32 s14, s92, s38
; SI-NEXT: s_mov_b32 s15, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[38:39], 16
-; SI-NEXT: s_or_b32 s6, s21, s54
+; SI-NEXT: s_or_b32 s12, s93, s48
+; SI-NEXT: s_mov_b32 s13, s49
+; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
+; SI-NEXT: s_or_b32 s8, s95, s52
+; SI-NEXT: s_mov_b32 s9, s53
+; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
+; SI-NEXT: s_or_b32 s6, vcc_lo, s54
; SI-NEXT: s_mov_b32 s7, s55
; SI-NEXT: s_lshr_b64 s[54:55], s[54:55], 16
-; SI-NEXT: s_or_b32 s4, s22, s64
+; SI-NEXT: s_or_b32 s4, vcc_hi, s64
; SI-NEXT: s_mov_b32 s5, s65
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 16
-; SI-NEXT: s_lshr_b32 s53, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_lshr_b32 s51, s16, 16
+; SI-NEXT: v_readlane_b32 s16, v46, 4
; SI-NEXT: s_or_b32 s46, s79, s40
; SI-NEXT: s_mov_b32 s47, s41
+; SI-NEXT: s_or_b32 s44, s89, s42
+; SI-NEXT: s_mov_b32 s45, s43
+; SI-NEXT: s_or_b32 s42, s17, s34
+; SI-NEXT: s_mov_b32 s43, s35
+; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
; SI-NEXT: s_or_b32 s40, s91, s36
; SI-NEXT: s_mov_b32 s41, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[36:37], 16
-; SI-NEXT: s_lshr_b32 s55, s16, 16
-; SI-NEXT: s_lshr_b32 s65, s86, 16
-; SI-NEXT: s_lshr_b32 s35, s29, 16
-; SI-NEXT: s_mov_b32 s94, s66
+; SI-NEXT: s_lshr_b32 s49, s81, 16
+; SI-NEXT: s_lshr_b32 s53, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s84, 16
+; SI-NEXT: s_lshr_b32 s65, s69, 16
+; SI-NEXT: s_lshr_b32 s69, s70, 16
; SI-NEXT: s_lshr_b32 s79, s28, 16
-; SI-NEXT: v_readlane_b32 s28, v46, 3
-; SI-NEXT: v_readlane_b32 vcc_hi, v46, 2
+; SI-NEXT: v_readlane_b32 s28, v46, 7
+; SI-NEXT: v_readlane_b32 vcc_hi, v46, 6
; SI-NEXT: s_lshr_b32 s89, s19, 16
; SI-NEXT: s_lshr_b32 s91, s24, 16
-; SI-NEXT: v_readlane_b32 s24, v46, 4
+; SI-NEXT: v_readlane_b32 s24, v46, 8
; SI-NEXT: s_lshr_b32 s93, s27, 16
-; SI-NEXT: v_readlane_b32 s27, v46, 6
+; SI-NEXT: v_readlane_b32 s27, v46, 10
; SI-NEXT: s_lshr_b32 s95, s18, 16
; SI-NEXT: s_lshr_b32 s31, s26, 16
; SI-NEXT: s_lshr_b32 s17, s20, 16
-; SI-NEXT: v_readlane_b32 s20, v46, 1
+; SI-NEXT: v_readlane_b32 s20, v46, 5
; SI-NEXT: s_lshr_b32 s92, s78, 16
-; SI-NEXT: s_lshr_b32 s78, s88, 16
-; SI-NEXT: v_readlane_b32 s21, v47, 47
+; SI-NEXT: s_lshr_b32 s68, s88, 16
+; SI-NEXT: v_readlane_b32 s21, v47, 57
; SI-NEXT: s_mov_b32 s26, s23
-; SI-NEXT: s_mov_b32 s22, s25
-; SI-NEXT: v_readlane_b32 s25, v46, 5
+; SI-NEXT: v_readlane_b32 s25, v46, 9
; SI-NEXT: s_lshr_b32 s39, s90, 16
; SI-NEXT: s_mov_b64 s[66:67], 0
; SI-NEXT: s_branch .LBB97_3
; SI-NEXT: .LBB97_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s71, s47
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: s_mov_b64 s[66:67], -1
+; SI-NEXT: v_writelane_b32 v47, s4, 47
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr49
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr30
; SI-NEXT: ; implicit-def: $sgpr51
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr70
; SI-NEXT: ; implicit-def: $sgpr65
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr69
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr34
@@ -194821,41 +195057,62 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: ; implicit-def: $sgpr92
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr54
-; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s5, 56
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB97_3: ; %Flow
; SI-NEXT: s_and_b64 s[18:19], s[66:67], exec
; SI-NEXT: s_cselect_b32 s16, 1, 0
; SI-NEXT: s_cmp_lg_u32 s16, 1
+; SI-NEXT: v_readlane_b32 s16, v47, 0
+; SI-NEXT: v_readlane_b32 s18, v47, 7
+; SI-NEXT: v_readlane_b32 s19, v47, 8
+; SI-NEXT: v_readlane_b32 s88, v47, 9
+; SI-NEXT: v_readlane_b32 s90, v47, 10
+; SI-NEXT: v_readlane_b32 s78, v47, 11
+; SI-NEXT: v_readlane_b32 s94, v47, 16
+; SI-NEXT: v_readlane_b32 vcc_lo, v47, 27
+; SI-NEXT: v_readlane_b32 s23, v47, 30
+; SI-NEXT: v_readlane_b32 s29, v47, 36
+; SI-NEXT: v_readlane_b32 s37, v47, 40
+; SI-NEXT: v_readlane_b32 s35, v47, 41
+; SI-NEXT: v_readlane_b32 s66, v47, 42
; SI-NEXT: s_cbranch_scc1 .LBB97_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_readlane_b32 s4, v47, 32
+; SI-NEXT: v_readlane_b32 s4, v47, 28
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: v_readlane_b32 s5, v47, 27
-; SI-NEXT: v_readlane_b32 s6, v47, 17
+; SI-NEXT: v_readlane_b32 s5, v47, 23
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_add_i32 s6, s80, 3
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 15
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 24
+; SI-NEXT: s_lshl_b32 s5, s71, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s5, s5, s6
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 2
-; SI-NEXT: s_add_i32 s5, s5, 3
-; SI-NEXT: v_readlane_b32 s6, v56, 62
-; SI-NEXT: v_readlane_b32 s7, v56, 55
+; SI-NEXT: s_add_i32 s5, s16, 3
+; SI-NEXT: v_readlane_b32 s6, v56, 60
+; SI-NEXT: v_readlane_b32 s7, v56, 53
; SI-NEXT: s_and_b32 s5, s5, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 8
; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v56, 52
+; SI-NEXT: v_readlane_b32 s6, v56, 50
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 24
; SI-NEXT: s_lshl_b32 s7, s7, 16
@@ -194863,15 +195120,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v47, 41
+; SI-NEXT: v_readlane_b32 s6, v47, 37
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 35
-; SI-NEXT: v_readlane_b32 s8, v47, 30
+; SI-NEXT: v_readlane_b32 s7, v47, 31
+; SI-NEXT: v_readlane_b32 s8, v47, 26
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_add_i32 s8, s8, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 29
+; SI-NEXT: v_readlane_b32 s7, v47, 25
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_lshl_b32 s8, s8, 16
@@ -194879,14 +195136,13 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s7, s7, s8
; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 12
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 6
+; SI-NEXT: s_add_i32 s7, s90, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 4
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s21, 8
; SI-NEXT: s_add_i32 s9, s9, 3
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 1
+; SI-NEXT: v_readlane_b32 s8, v56, 63
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_lshl_b32 s9, s9, 16
@@ -194894,15 +195150,13 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s8, s8, s9
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 44
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 43
-; SI-NEXT: v_readlane_b32 s10, v47, 40
+; SI-NEXT: s_add_i32 s8, s37, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 39
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
-; SI-NEXT: s_add_i32 s10, s10, 3
+; SI-NEXT: s_add_i32 s10, s29, 3
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 37
+; SI-NEXT: v_readlane_b32 s9, v47, 33
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 24
; SI-NEXT: s_lshl_b32 s10, s10, 16
@@ -194910,31 +195164,27 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_and_b32 s8, s8, 0xffff
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 25
+; SI-NEXT: v_readlane_b32 s9, v47, 21
; SI-NEXT: s_add_i32 s9, s9, 3
-; SI-NEXT: v_readlane_b32 s10, v47, 21
-; SI-NEXT: v_readlane_b32 s11, v47, 11
+; SI-NEXT: v_readlane_b32 s10, v47, 17
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 8
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s11, s88, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 10
; SI-NEXT: s_and_b32 s11, s11, 0xff
-; SI-NEXT: s_lshl_b32 s10, s10, 24
+; SI-NEXT: s_lshl_b32 s10, s19, 24
; SI-NEXT: s_lshl_b32 s11, s11, 16
; SI-NEXT: s_addk_i32 s9, 0x300
; SI-NEXT: s_or_b32 s10, s10, s11
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 46
-; SI-NEXT: s_add_i32 s10, s10, 3
-; SI-NEXT: v_readlane_b32 s11, v47, 45
-; SI-NEXT: v_readlane_b32 s12, v47, 42
+; SI-NEXT: s_add_i32 s10, s66, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 38
; SI-NEXT: s_and_b32 s10, s10, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 8
+; SI-NEXT: s_lshl_b32 s11, s35, 8
; SI-NEXT: s_add_i32 s12, s12, 3
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 39
+; SI-NEXT: v_readlane_b32 s11, v47, 35
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 24
; SI-NEXT: s_lshl_b32 s12, s12, 16
@@ -194942,15 +195192,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s11, s11, s12
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 31
-; SI-NEXT: s_add_i32 s11, s11, 3
-; SI-NEXT: v_readlane_b32 s12, v47, 28
-; SI-NEXT: v_readlane_b32 s13, v47, 22
+; SI-NEXT: s_add_i32 s11, vcc_lo, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 24
+; SI-NEXT: v_readlane_b32 s13, v47, 18
; SI-NEXT: s_and_b32 s11, s11, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 8
; SI-NEXT: s_add_i32 s13, s13, 3
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 16
+; SI-NEXT: v_readlane_b32 s12, v47, 13
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 24
; SI-NEXT: s_lshl_b32 s13, s13, 16
@@ -194958,47 +195207,44 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s12, s12, s13
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 38
+; SI-NEXT: v_readlane_b32 s12, v47, 34
; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: v_readlane_b32 s13, v47, 36
-; SI-NEXT: v_readlane_b32 s14, v47, 33
+; SI-NEXT: v_readlane_b32 s13, v47, 32
+; SI-NEXT: v_readlane_b32 s14, v47, 29
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s13, s13, 8
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 34
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s13, s13, 24
+; SI-NEXT: s_lshl_b32 s13, s23, 24
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_addk_i32 s12, 0x300
; SI-NEXT: s_or_b32 s13, s13, s14
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 26
+; SI-NEXT: v_readlane_b32 s13, v47, 22
; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s14, v47, 24
-; SI-NEXT: v_readlane_b32 s15, v47, 19
+; SI-NEXT: v_readlane_b32 s14, v47, 20
+; SI-NEXT: v_readlane_b32 s15, v47, 15
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 8
; SI-NEXT: s_add_i32 s15, s15, 3
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 20
; SI-NEXT: s_and_b32 s15, s15, 0xff
-; SI-NEXT: s_lshl_b32 s14, s14, 24
+; SI-NEXT: s_lshl_b32 s14, s94, 24
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_addk_i32 s13, 0x300
; SI-NEXT: s_or_b32 s14, s14, s15
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 23
+; SI-NEXT: v_readlane_b32 s14, v47, 19
; SI-NEXT: s_add_i32 s14, s14, 3
-; SI-NEXT: v_readlane_b32 s15, v47, 18
-; SI-NEXT: v_readlane_b32 s16, v47, 13
+; SI-NEXT: v_readlane_b32 s15, v47, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 8
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s78, 3
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 14
+; SI-NEXT: v_readlane_b32 s15, v47, 12
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 24
; SI-NEXT: s_lshl_b32 s16, s16, 16
@@ -195006,15 +195252,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s15, s15, s16
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 9
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s16, v47, 8
-; SI-NEXT: v_readlane_b32 s17, v47, 4
+; SI-NEXT: s_add_i32 s15, s18, 3
+; SI-NEXT: v_readlane_b32 s16, v47, 6
+; SI-NEXT: v_readlane_b32 s17, v47, 2
; SI-NEXT: s_and_b32 s15, s15, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 8
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 5
+; SI-NEXT: v_readlane_b32 s16, v47, 3
; SI-NEXT: s_and_b32 s17, s17, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 24
; SI-NEXT: s_lshl_b32 s17, s17, 16
@@ -195022,15 +195267,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 7
+; SI-NEXT: v_readlane_b32 s16, v47, 5
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v47, 3
-; SI-NEXT: v_readlane_b32 s18, v56, 63
+; SI-NEXT: v_readlane_b32 s17, v47, 1
+; SI-NEXT: v_readlane_b32 s18, v56, 61
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v47, 0
+; SI-NEXT: v_readlane_b32 s17, v56, 62
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195039,15 +195284,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s40, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 61
+; SI-NEXT: v_readlane_b32 s16, v56, 59
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 60
-; SI-NEXT: v_readlane_b32 s18, v56, 57
+; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s18, v56, 55
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s17, v56, 56
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195056,15 +195301,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s41, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 59
+; SI-NEXT: v_readlane_b32 s16, v56, 57
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 56
-; SI-NEXT: v_readlane_b32 s18, v56, 53
+; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s18, v56, 51
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s17, v56, 52
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195073,15 +195318,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 51
+; SI-NEXT: v_readlane_b32 s16, v56, 49
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 50
-; SI-NEXT: v_readlane_b32 s18, v56, 46
+; SI-NEXT: v_readlane_b32 s17, v56, 48
+; SI-NEXT: v_readlane_b32 s18, v56, 44
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 47
+; SI-NEXT: v_readlane_b32 s17, v56, 45
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195090,15 +195335,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s43, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 49
+; SI-NEXT: v_readlane_b32 s16, v56, 47
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 48
-; SI-NEXT: v_readlane_b32 s18, v56, 44
+; SI-NEXT: v_readlane_b32 s17, v56, 46
+; SI-NEXT: v_readlane_b32 s18, v56, 42
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 45
+; SI-NEXT: v_readlane_b32 s17, v56, 43
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195107,15 +195352,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s44, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 43
+; SI-NEXT: v_readlane_b32 s16, v56, 41
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 42
-; SI-NEXT: v_readlane_b32 s18, v56, 38
+; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s18, v56, 36
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 39
+; SI-NEXT: v_readlane_b32 s17, v56, 37
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195124,14 +195369,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s45, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 41
+; SI-NEXT: v_readlane_b32 s16, v56, 39
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s17, v56, 38
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s87, 3
+; SI-NEXT: s_add_i32 s18, s85, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 37
+; SI-NEXT: v_readlane_b32 s17, v56, 35
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195143,7 +195388,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s16, s28, 3
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s25, 8
-; SI-NEXT: s_add_i32 s18, s85, 3
+; SI-NEXT: s_add_i32 s18, s30, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
@@ -195219,7 +195464,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_add_i32 s59, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s76, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 29
+; SI-NEXT: v_readlane_b32 s17, v56, 28
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s97, 3
@@ -195232,8 +195477,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s60, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 28
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s86, 3
; SI-NEXT: v_readlane_b32 s17, v56, 27
; SI-NEXT: v_readlane_b32 s18, v56, 26
; SI-NEXT: s_and_b32 s16, s16, 0xff
@@ -195250,7 +195494,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s61, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s77, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 34
+; SI-NEXT: v_readlane_b32 s17, v56, 32
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s99, 3
@@ -195263,15 +195507,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s62, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 33
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 32
-; SI-NEXT: v_readlane_b32 s18, v56, 31
+; SI-NEXT: s_add_i32 s16, s87, 3
+; SI-NEXT: v_readlane_b32 s17, v56, 31
+; SI-NEXT: v_readlane_b32 s18, v56, 30
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 30
+; SI-NEXT: v_readlane_b32 s17, v56, 29
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195299,12 +195542,12 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readlane_b32 s16, v56, 1
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: v_readlane_b32 s17, v56, 0
-; SI-NEXT: v_readlane_b32 s18, v56, 36
+; SI-NEXT: v_readlane_b32 s18, v56, 34
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 35
+; SI-NEXT: v_readlane_b32 s17, v56, 33
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195344,6 +195587,25 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s18, s18, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s75, s16, 0x3000000
+; SI-NEXT: s_lshr_b64 s[16:17], s[72:73], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 43
+; SI-NEXT: v_writelane_b32 v47, s17, 44
+; SI-NEXT: s_lshr_b64 s[16:17], s[62:63], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 45
+; SI-NEXT: v_writelane_b32 v47, s17, 46
+; SI-NEXT: s_lshr_b64 s[16:17], s[60:61], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 47
+; SI-NEXT: v_writelane_b32 v47, s17, 48
+; SI-NEXT: s_lshr_b64 s[16:17], s[58:59], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 49
+; SI-NEXT: v_writelane_b32 v47, s17, 50
+; SI-NEXT: s_lshr_b64 s[16:17], s[56:57], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 51
+; SI-NEXT: v_writelane_b32 v47, s17, 52
+; SI-NEXT: s_lshr_b64 s[16:17], s[46:47], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 53
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -195356,11 +195618,9 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s13, s13, 0x3000000
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s75, s16, 0x3000000
-; SI-NEXT: s_lshr_b64 s[30:31], s[72:73], 16
-; SI-NEXT: s_lshr_b64 s[94:95], s[62:63], 16
-; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 54
+; SI-NEXT: s_lshr_b64 s[16:17], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 55
; SI-NEXT: s_lshr_b64 s[38:39], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[50:51], s[10:11], 16
@@ -195368,18 +195628,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshr_b64 s[54:55], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[4:5], 16
; SI-NEXT: s_lshr_b64 s[82:83], s[74:75], 16
-; SI-NEXT: s_lshr_b64 vcc, s[60:61], 16
-; SI-NEXT: s_lshr_b64 s[70:71], s[58:59], 16
-; SI-NEXT: s_lshr_b64 s[80:81], s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[84:85], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[68:69], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 56
+; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[40:41], 16
; SI-NEXT: s_lshr_b32 s49, s75, 16
; SI-NEXT: s_lshr_b32 s51, s73, 16
; SI-NEXT: s_lshr_b32 s53, s63, 16
; SI-NEXT: s_lshr_b32 s55, s61, 16
; SI-NEXT: s_lshr_b32 s65, s59, 16
-; SI-NEXT: s_lshr_b32 s35, s57, 16
+; SI-NEXT: s_lshr_b32 s69, s57, 16
; SI-NEXT: s_lshr_b32 s79, s47, 16
; SI-NEXT: s_lshr_b32 s89, s45, 16
; SI-NEXT: s_lshr_b32 s91, s43, 16
@@ -195388,7 +195645,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: s_lshr_b32 s17, s11, 16
; SI-NEXT: s_lshr_b32 s92, s9, 16
-; SI-NEXT: s_lshr_b32 s78, s7, 16
+; SI-NEXT: s_lshr_b32 s68, s7, 16
; SI-NEXT: s_lshr_b32 s39, s5, 16
; SI-NEXT: .LBB97_5: ; %end
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -195401,45 +195658,57 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s16, s19
; SI-NEXT: s_and_b32 s19, s75, 0xffff
; SI-NEXT: s_lshl_b32 s20, s49, 16
+; SI-NEXT: v_readlane_b32 s22, v47, 43
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s20, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s30, 16
+; SI-NEXT: s_lshl_b32 s21, s22, 16
+; SI-NEXT: v_readlane_b32 s23, v47, 44
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_and_b32 s21, s73, 0xffff
; SI-NEXT: s_lshl_b32 s22, s51, 16
+; SI-NEXT: v_readlane_b32 s24, v47, 45
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s22, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s94, 16
+; SI-NEXT: s_lshl_b32 s23, s24, 16
+; SI-NEXT: v_readlane_b32 s25, v47, 46
; SI-NEXT: s_or_b32 s22, s22, s23
; SI-NEXT: s_and_b32 s23, s63, 0xffff
; SI-NEXT: s_lshl_b32 s24, s53, 16
+; SI-NEXT: v_readlane_b32 s26, v47, 47
; SI-NEXT: s_or_b32 s23, s23, s24
; SI-NEXT: s_and_b32 s24, s60, 0xffff
-; SI-NEXT: s_lshl_b32 s25, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s25, s26, 16
+; SI-NEXT: v_readlane_b32 s27, v47, 48
; SI-NEXT: s_or_b32 s24, s24, s25
; SI-NEXT: s_and_b32 s25, s61, 0xffff
; SI-NEXT: s_lshl_b32 s26, s55, 16
+; SI-NEXT: v_readlane_b32 s28, v47, 49
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s26, s58, 0xffff
-; SI-NEXT: s_lshl_b32 s27, s70, 16
+; SI-NEXT: s_lshl_b32 s27, s28, 16
+; SI-NEXT: v_readlane_b32 s29, v47, 50
; SI-NEXT: s_or_b32 s26, s26, s27
; SI-NEXT: s_and_b32 s27, s59, 0xffff
; SI-NEXT: s_lshl_b32 s28, s65, 16
+; SI-NEXT: v_readlane_b32 s58, v47, 51
; SI-NEXT: s_or_b32 s27, s27, s28
; SI-NEXT: s_and_b32 s28, s56, 0xffff
-; SI-NEXT: s_lshl_b32 s29, s80, 16
+; SI-NEXT: s_lshl_b32 s29, s58, 16
; SI-NEXT: s_or_b32 s28, s28, s29
; SI-NEXT: s_and_b32 s29, s57, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_lshl_b32 s56, s69, 16
; SI-NEXT: s_or_b32 s29, s29, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 53
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s84, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: v_readlane_b32 s57, v47, 54
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s47, s47, 0xffff
; SI-NEXT: s_lshl_b32 s56, s79, 16
; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 55
; SI-NEXT: s_and_b32 s44, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s68, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_or_b32 s44, s44, s56
; SI-NEXT: s_and_b32 s45, s45, 0xffff
; SI-NEXT: s_lshl_b32 s56, s89, 16
@@ -195477,7 +195746,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s56, s48, 16
; SI-NEXT: s_or_b32 s6, s6, s17
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s78, 16
+; SI-NEXT: s_lshl_b32 s17, s68, 16
; SI-NEXT: s_or_b32 s12, s12, s56
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s56, s31, 16
@@ -195493,6 +195762,8 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s10, s10, s56
; SI-NEXT: s_or_b32 s5, s5, s17
; SI-NEXT: v_readlane_b32 s30, v45, 34
+; SI-NEXT: v_readlane_b32 s59, v47, 52
+; SI-NEXT: v_readlane_b32 s57, v47, 56
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s19
; SI-NEXT: v_mov_b32_e32 v2, s20
@@ -203565,7 +203836,7 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_or_b32 s8, s4, s5
; SI-NEXT: s_and_b32 s4, s94, 0xffff
; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: v_writelane_b32 v21, s18, 42
+; SI-NEXT: v_writelane_b32 v21, s18, 41
; SI-NEXT: s_lshr_b32 s18, s93, 8
; SI-NEXT: s_or_b32 s9, s4, s5
; SI-NEXT: s_and_b32 s4, s38, 0xffff
@@ -203603,7 +203874,7 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b32 s96, s5, 8
; SI-NEXT: v_writelane_b32 v21, s96, 62
; SI-NEXT: s_bfe_u32 s18, s53, 0x80008
-; SI-NEXT: v_writelane_b32 v21, s18, 41
+; SI-NEXT: v_writelane_b32 v21, s18, 42
; SI-NEXT: s_bfe_u32 s18, s51, 0x80008
; SI-NEXT: v_writelane_b32 v21, s18, 43
; SI-NEXT: s_bfe_u32 s18, s49, 0x80008
@@ -203969,9 +204240,11 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB99_3: ; %Flow
; SI-NEXT: s_and_b64 s[96:97], s[96:97], exec
-; SI-NEXT: s_cselect_b32 s96, 1, 0
; SI-NEXT: v_writelane_b32 v21, s54, 37
+; SI-NEXT: s_cselect_b32 s96, 1, 0
+; SI-NEXT: v_writelane_b32 v21, s55, 38
; SI-NEXT: s_cmp_lg_u32 s96, 1
+; SI-NEXT: s_mov_b32 s54, s18
; SI-NEXT: s_mov_b32 s96, s72
; SI-NEXT: s_mov_b32 s72, s76
; SI-NEXT: s_mov_b32 s76, s78
@@ -203981,13 +204254,12 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_mov_b32 s68, s20
; SI-NEXT: s_mov_b32 s20, s70
; SI-NEXT: v_readlane_b32 s70, v22, 47
-; SI-NEXT: v_writelane_b32 v21, s55, 38
+; SI-NEXT: v_readlane_b32 s18, v22, 29
; SI-NEXT: v_readlane_b32 s71, v22, 48
-; SI-NEXT: s_mov_b32 s54, s18
+; SI-NEXT: v_readlane_b32 s19, v22, 30
; SI-NEXT: v_readlane_b32 s71, v21, 39
; SI-NEXT: v_readlane_b32 s97, v21, 40
-; SI-NEXT: v_readlane_b32 s18, v22, 29
-; SI-NEXT: v_readlane_b32 s19, v22, 30
+; SI-NEXT: v_readlane_b32 s19, v21, 41
; SI-NEXT: s_cbranch_scc1 .LBB99_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s50, s50, 3
@@ -204283,8 +204555,6 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v21, s16, 37
; SI-NEXT: v_writelane_b32 v21, s17, 38
; SI-NEXT: s_lshr_b32 s16, s65, 24
-; SI-NEXT: v_writelane_b32 v21, s16, 41
-; SI-NEXT: s_lshr_b32 s16, s65, 8
; SI-NEXT: v_writelane_b32 v21, s16, 42
; SI-NEXT: s_lshr_b32 s16, s93, 24
; SI-NEXT: v_writelane_b32 v21, s16, 43
@@ -204329,6 +204599,7 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_add_i32 s8, s8, 0x30000
; SI-NEXT: v_writelane_b32 v22, s16, 0
; SI-NEXT: s_lshr_b32 s16, s7, 8
+; SI-NEXT: s_lshr_b64 s[18:19], s[64:65], 24
; SI-NEXT: s_lshr_b64 s[70:71], s[58:59], 24
; SI-NEXT: s_lshr_b64 s[20:21], s[56:57], 8
; SI-NEXT: s_lshr_b64 vcc, s[8:9], 24
@@ -204341,8 +204612,8 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b64 s[96:97], s[4:5], 8
; SI-NEXT: v_writelane_b32 v21, s16, 60
; SI-NEXT: s_lshr_b32 s16, s5, 24
-; SI-NEXT: s_lshr_b64 s[18:19], s[64:65], 24
; SI-NEXT: s_lshr_b32 s53, s65, 16
+; SI-NEXT: s_lshr_b32 s19, s65, 8
; SI-NEXT: s_lshr_b32 s51, s93, 16
; SI-NEXT: s_lshr_b32 s49, s63, 16
; SI-NEXT: s_lshr_b32 s39, s59, 16
@@ -204382,13 +204653,12 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s18, s17
; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: v_readlane_b32 s17, v21, 42
; SI-NEXT: v_mov_b32_e32 v1, s16
; SI-NEXT: s_and_b32 s16, s65, 0xff
-; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_lshl_b32 s17, s19, 8
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s53, 0xff
-; SI-NEXT: v_readlane_b32 s18, v21, 41
+; SI-NEXT: v_readlane_b32 s18, v21, 42
; SI-NEXT: s_lshl_b32 s17, s17, 16
; SI-NEXT: s_lshl_b32 s18, s18, 24
; SI-NEXT: s_or_b32 s17, s18, s17
@@ -217530,22 +217800,22 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_writelane_b32 v63, s99, 33
; SI-NEXT: v_writelane_b32 v63, s30, 34
; SI-NEXT: v_writelane_b32 v63, s31, 35
-; SI-NEXT: s_lshr_b32 s6, s16, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
; SI-NEXT: ; implicit-def: $vgpr62 : SGPR spill to VGPR lane
-; SI-NEXT: s_lshr_b32 s7, s17, 16
-; SI-NEXT: v_writelane_b32 v62, s6, 0
-; SI-NEXT: s_lshr_b32 s8, s18, 16
-; SI-NEXT: v_writelane_b32 v62, s7, 1
-; SI-NEXT: s_lshr_b32 s9, s19, 16
-; SI-NEXT: v_writelane_b32 v62, s8, 2
-; SI-NEXT: s_lshr_b32 s10, s20, 16
-; SI-NEXT: v_writelane_b32 v62, s9, 3
-; SI-NEXT: s_lshr_b32 s11, s21, 16
-; SI-NEXT: v_writelane_b32 v62, s10, 4
-; SI-NEXT: s_lshr_b32 s12, s22, 16
-; SI-NEXT: v_readfirstlane_b32 s49, v17
-; SI-NEXT: v_readfirstlane_b32 s37, v16
-; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s8, s17, 16
+; SI-NEXT: v_writelane_b32 v62, s7, 0
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: v_writelane_b32 v62, s8, 1
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: v_writelane_b32 v62, s9, 2
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: v_writelane_b32 v62, s10, 3
+; SI-NEXT: s_lshr_b32 s30, s21, 16
+; SI-NEXT: v_writelane_b32 v62, s11, 4
+; SI-NEXT: s_lshr_b32 s31, s22, 16
+; SI-NEXT: v_readfirstlane_b32 s4, v17
+; SI-NEXT: v_readfirstlane_b32 s12, v16
+; SI-NEXT: v_readfirstlane_b32 s70, v15
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v14
; SI-NEXT: v_readfirstlane_b32 s94, v13
; SI-NEXT: v_readfirstlane_b32 s92, v12
@@ -217561,17 +217831,17 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readfirstlane_b32 s56, v2
; SI-NEXT: v_readfirstlane_b32 s46, v1
; SI-NEXT: v_readfirstlane_b32 s44, v0
-; SI-NEXT: v_writelane_b32 v62, s11, 5
+; SI-NEXT: v_writelane_b32 v62, s30, 5
; SI-NEXT: s_lshr_b32 s43, s29, 16
; SI-NEXT: s_lshr_b32 s42, s28, 16
; SI-NEXT: s_lshr_b32 s41, s27, 16
; SI-NEXT: s_lshr_b32 s40, s26, 16
; SI-NEXT: s_lshr_b32 s15, s25, 16
; SI-NEXT: s_lshr_b32 s14, s24, 16
-; SI-NEXT: s_lshr_b32 s13, s23, 16
-; SI-NEXT: s_lshr_b32 s52, s49, 16
-; SI-NEXT: s_lshr_b32 s48, s37, 16
-; SI-NEXT: s_lshr_b32 s35, s4, 16
+; SI-NEXT: s_lshr_b32 s34, s23, 16
+; SI-NEXT: s_lshr_b32 s5, s4, 16
+; SI-NEXT: s_lshr_b32 s13, s12, 16
+; SI-NEXT: s_lshr_b32 s71, s70, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
; SI-NEXT: s_lshr_b32 s93, s92, 16
@@ -217587,161 +217857,171 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: s_lshr_b32 s57, s56, 16
; SI-NEXT: s_lshr_b32 s47, s46, 16
; SI-NEXT: s_lshr_b32 s45, s44, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_writelane_b32 v62, s12, 6
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_writelane_b32 v62, s13, 7
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v62, s31, 6
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v62, s34, 7
; SI-NEXT: s_cbranch_scc0 .LBB103_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_lshl_b32 s5, s16, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 33
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 32
-; SI-NEXT: s_lshl_b32 s5, s17, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 31
-; SI-NEXT: s_lshl_b32 s5, s7, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 30
-; SI-NEXT: s_lshl_b32 s5, s18, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 29
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 28
-; SI-NEXT: s_lshl_b32 s5, s19, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 27
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 26
-; SI-NEXT: s_lshl_b32 s5, s20, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 25
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 24
-; SI-NEXT: s_lshl_b32 s5, s21, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 23
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 22
-; SI-NEXT: s_lshl_b32 s5, s22, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 21
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 20
-; SI-NEXT: s_lshl_b32 s5, s23, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 19
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 18
-; SI-NEXT: s_lshl_b32 s5, s24, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 17
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 16
-; SI-NEXT: s_lshl_b32 s5, s25, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 15
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 14
-; SI-NEXT: s_lshl_b32 s5, s26, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 13
-; SI-NEXT: s_lshl_b32 s5, s40, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 12
-; SI-NEXT: s_lshl_b32 s5, s27, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 11
-; SI-NEXT: s_lshl_b32 s5, s41, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 10
-; SI-NEXT: s_lshl_b32 s5, s28, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 9
-; SI-NEXT: s_lshl_b32 s5, s42, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 8
-; SI-NEXT: s_lshl_b32 s87, s29, 16
-; SI-NEXT: s_lshl_b32 s96, s43, 16
-; SI-NEXT: s_lshl_b32 s97, s44, 16
-; SI-NEXT: s_lshl_b32 s98, s45, 16
-; SI-NEXT: s_lshl_b32 s99, s46, 16
-; SI-NEXT: s_lshl_b32 s30, s47, 16
-; SI-NEXT: s_lshl_b32 s31, s56, 16
-; SI-NEXT: s_lshl_b32 s36, s57, 16
-; SI-NEXT: s_lshl_b32 s38, s58, 16
-; SI-NEXT: s_lshl_b32 s39, s59, 16
-; SI-NEXT: s_lshl_b32 s50, s60, 16
-; SI-NEXT: s_lshl_b32 s51, s61, 16
-; SI-NEXT: s_lshl_b32 s53, s62, 16
-; SI-NEXT: s_lshl_b32 s54, s63, 16
-; SI-NEXT: s_lshl_b32 s55, s72, 16
-; SI-NEXT: s_lshl_b32 s64, s73, 16
+; SI-NEXT: s_lshl_b32 s6, s16, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 33
+; SI-NEXT: s_lshl_b32 s6, s7, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 32
+; SI-NEXT: s_lshl_b32 s6, s17, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 31
+; SI-NEXT: s_lshl_b32 s6, s8, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 30
+; SI-NEXT: s_lshl_b32 s6, s18, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 29
+; SI-NEXT: s_lshl_b32 s6, s9, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 28
+; SI-NEXT: s_lshl_b32 s6, s19, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 27
+; SI-NEXT: s_lshl_b32 s6, s10, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 26
+; SI-NEXT: s_lshl_b32 s6, s20, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 25
+; SI-NEXT: s_lshl_b32 s6, s11, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 24
+; SI-NEXT: s_lshl_b32 s6, s21, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 23
+; SI-NEXT: s_lshl_b32 s6, s30, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 22
+; SI-NEXT: s_lshl_b32 s6, s22, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 21
+; SI-NEXT: s_lshl_b32 s6, s31, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 20
+; SI-NEXT: s_lshl_b32 s6, s23, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 19
+; SI-NEXT: s_lshl_b32 s6, s34, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 18
+; SI-NEXT: s_lshl_b32 s6, s24, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 17
+; SI-NEXT: s_lshl_b32 s6, s14, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 16
+; SI-NEXT: s_lshl_b32 s6, s25, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 15
+; SI-NEXT: s_lshl_b32 s6, s15, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 14
+; SI-NEXT: s_lshl_b32 s6, s26, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 13
+; SI-NEXT: s_lshl_b32 s6, s40, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 12
+; SI-NEXT: s_lshl_b32 s6, s27, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 11
+; SI-NEXT: s_lshl_b32 s6, s41, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 10
+; SI-NEXT: s_lshl_b32 s6, s28, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 9
+; SI-NEXT: s_lshl_b32 s6, s42, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 8
+; SI-NEXT: s_lshl_b32 s82, s29, 16
+; SI-NEXT: s_lshl_b32 s83, s43, 16
+; SI-NEXT: s_lshl_b32 s84, s44, 16
+; SI-NEXT: s_lshl_b32 s85, s45, 16
+; SI-NEXT: s_lshl_b32 s86, s46, 16
+; SI-NEXT: s_lshl_b32 s87, s47, 16
+; SI-NEXT: s_lshl_b32 s96, s56, 16
+; SI-NEXT: s_lshl_b32 s98, s57, 16
+; SI-NEXT: s_lshl_b32 s99, s58, 16
+; SI-NEXT: s_lshl_b32 s31, s59, 16
+; SI-NEXT: s_lshl_b32 s34, s60, 16
+; SI-NEXT: s_lshl_b32 s36, s61, 16
+; SI-NEXT: s_lshl_b32 s37, s62, 16
+; SI-NEXT: s_lshl_b32 s39, s63, 16
+; SI-NEXT: s_lshl_b32 s48, s72, 16
+; SI-NEXT: s_lshl_b32 s50, s73, 16
; SI-NEXT: s_lshl_b32 s6, s74, 16
-; SI-NEXT: s_lshl_b32 s65, s75, 16
+; SI-NEXT: s_lshl_b32 s52, s75, 16
; SI-NEXT: s_lshl_b32 s7, s76, 16
-; SI-NEXT: s_lshl_b32 s66, s77, 16
-; SI-NEXT: s_lshl_b32 s67, s78, 16
-; SI-NEXT: s_lshl_b32 s68, s79, 16
+; SI-NEXT: s_lshl_b32 s53, s77, 16
+; SI-NEXT: s_lshl_b32 s54, s78, 16
+; SI-NEXT: s_lshl_b32 s55, s79, 16
; SI-NEXT: s_lshl_b32 s8, s88, 16
-; SI-NEXT: s_lshl_b32 s69, s89, 16
-; SI-NEXT: s_lshl_b32 s70, s90, 16
+; SI-NEXT: s_lshl_b32 s64, s89, 16
+; SI-NEXT: s_lshl_b32 s65, s90, 16
; SI-NEXT: s_lshl_b32 s9, s91, 16
-; SI-NEXT: s_lshl_b32 s71, s92, 16
-; SI-NEXT: s_lshl_b32 s80, s93, 16
+; SI-NEXT: s_lshl_b32 s66, s92, 16
+; SI-NEXT: s_lshl_b32 s67, s93, 16
; SI-NEXT: s_lshl_b32 s10, s94, 16
-; SI-NEXT: s_lshl_b32 s81, s95, 16
-; SI-NEXT: s_lshl_b32 s82, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s68, s95, 16
+; SI-NEXT: s_lshl_b32 s69, vcc_lo, 16
; SI-NEXT: s_lshl_b32 s11, vcc_hi, 16
-; SI-NEXT: s_mov_b32 s34, s4
-; SI-NEXT: s_lshl_b32 s83, s4, 16
-; SI-NEXT: s_lshl_b32 s84, s35, 16
-; SI-NEXT: s_lshl_b32 s12, s37, 16
-; SI-NEXT: s_lshl_b32 s85, s48, 16
-; SI-NEXT: s_lshl_b32 s13, s49, 16
-; SI-NEXT: s_lshl_b32 s86, s52, 16
+; SI-NEXT: s_mov_b32 s30, s70
+; SI-NEXT: s_lshl_b32 s70, s70, 16
+; SI-NEXT: s_mov_b32 s97, s71
+; SI-NEXT: s_lshl_b32 s71, s71, 16
+; SI-NEXT: s_mov_b32 s35, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_mov_b32 s38, s13
+; SI-NEXT: s_lshl_b32 s80, s13, 16
+; SI-NEXT: s_mov_b32 s49, s4
+; SI-NEXT: s_lshl_b32 s13, s4, 16
+; SI-NEXT: s_mov_b32 s51, s5
+; SI-NEXT: s_lshl_b32 s81, s5, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB103_3
; SI-NEXT: .LBB103_2:
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s34, s4
+; SI-NEXT: s_mov_b32 s51, s5
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s49, s4
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s38, s13
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s35, s12
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s97, s71
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s30, s70
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr82
+; SI-NEXT: ; implicit-def: $sgpr83
+; SI-NEXT: ; implicit-def: $sgpr84
+; SI-NEXT: ; implicit-def: $sgpr85
+; SI-NEXT: ; implicit-def: $sgpr86
; SI-NEXT: ; implicit-def: $sgpr87
; SI-NEXT: ; implicit-def: $sgpr96
-; SI-NEXT: ; implicit-def: $sgpr97
; SI-NEXT: ; implicit-def: $sgpr98
; SI-NEXT: ; implicit-def: $sgpr99
-; SI-NEXT: ; implicit-def: $sgpr30
; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr36
-; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr37
; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr48
; SI-NEXT: ; implicit-def: $sgpr50
-; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr54
; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr8
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr66
; SI-NEXT: ; implicit-def: $sgpr67
+; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr8
; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr11
; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr81
-; SI-NEXT: ; implicit-def: $sgpr82
-; SI-NEXT: ; implicit-def: $sgpr11
-; SI-NEXT: ; implicit-def: $sgpr83
-; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr85
+; SI-NEXT: ; implicit-def: $sgpr80
; SI-NEXT: ; implicit-def: $sgpr13
-; SI-NEXT: ; implicit-def: $sgpr86
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr81
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
; SI-NEXT: ; implicit-def: $sgpr6
@@ -217785,9 +218065,9 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB103_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s52
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s51
; SI-NEXT: v_cvt_f32_f16_e32 v1, s49
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s48
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s38
; SI-NEXT: v_readlane_b32 s4, v62, 7
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -217801,10 +218081,10 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s37
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s35
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s97
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -217813,7 +218093,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v1, vcc_hi
@@ -218145,7 +218425,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 25
; SI-NEXT: v_mov_b32_e32 v38, s4
; SI-NEXT: v_readlane_b32 s4, v62, 26
-; SI-NEXT: v_mov_b32_e32 v0, s86
+; SI-NEXT: v_mov_b32_e32 v0, s81
; SI-NEXT: v_mov_b32_e32 v48, s4
; SI-NEXT: v_readlane_b32 s4, v62, 27
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
@@ -218155,7 +218435,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 28
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s85
+; SI-NEXT: v_mov_b32_e32 v0, s80
; SI-NEXT: v_mov_b32_e32 v5, s4
; SI-NEXT: v_readlane_b32 s4, v62, 29
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
@@ -218165,12 +218445,12 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 30
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s84
+; SI-NEXT: v_mov_b32_e32 v0, s71
; SI-NEXT: v_mov_b32_e32 v3, s4
; SI-NEXT: v_readlane_b32 s4, v62, 31
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s83
+; SI-NEXT: v_mov_b32_e32 v0, s70
; SI-NEXT: v_mov_b32_e32 v2, s4
; SI-NEXT: v_readlane_b32 s4, v62, 32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
@@ -218179,37 +218459,37 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_mov_b32_e32 v1, s4
; SI-NEXT: v_readlane_b32 s4, v62, 33
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v39, s82
-; SI-NEXT: v_mov_b32_e32 v51, s81
+; SI-NEXT: v_mov_b32_e32 v39, s69
+; SI-NEXT: v_mov_b32_e32 v51, s68
; SI-NEXT: v_mov_b32_e32 v49, s10
-; SI-NEXT: v_mov_b32_e32 v53, s80
-; SI-NEXT: v_mov_b32_e32 v52, s71
+; SI-NEXT: v_mov_b32_e32 v53, s67
+; SI-NEXT: v_mov_b32_e32 v52, s66
; SI-NEXT: v_mov_b32_e32 v55, s9
-; SI-NEXT: v_mov_b32_e32 v54, s70
-; SI-NEXT: v_mov_b32_e32 v41, s69
+; SI-NEXT: v_mov_b32_e32 v54, s65
+; SI-NEXT: v_mov_b32_e32 v41, s64
; SI-NEXT: v_mov_b32_e32 v40, s8
-; SI-NEXT: v_mov_b32_e32 v43, s68
-; SI-NEXT: v_mov_b32_e32 v42, s67
-; SI-NEXT: v_mov_b32_e32 v45, s66
+; SI-NEXT: v_mov_b32_e32 v43, s55
+; SI-NEXT: v_mov_b32_e32 v42, s54
+; SI-NEXT: v_mov_b32_e32 v45, s53
; SI-NEXT: v_mov_b32_e32 v44, s7
-; SI-NEXT: v_mov_b32_e32 v47, s65
+; SI-NEXT: v_mov_b32_e32 v47, s52
; SI-NEXT: v_mov_b32_e32 v46, s6
-; SI-NEXT: v_mov_b32_e32 v57, s64
-; SI-NEXT: v_mov_b32_e32 v56, s55
-; SI-NEXT: v_mov_b32_e32 v59, s54
-; SI-NEXT: v_mov_b32_e32 v58, s53
-; SI-NEXT: v_mov_b32_e32 v61, s51
-; SI-NEXT: v_mov_b32_e32 v60, s50
-; SI-NEXT: v_mov_b32_e32 v37, s39
-; SI-NEXT: v_mov_b32_e32 v36, s38
-; SI-NEXT: v_mov_b32_e32 v33, s36
-; SI-NEXT: v_mov_b32_e32 v32, s31
-; SI-NEXT: v_mov_b32_e32 v31, s30
-; SI-NEXT: v_mov_b32_e32 v30, s99
-; SI-NEXT: v_mov_b32_e32 v29, s98
-; SI-NEXT: v_mov_b32_e32 v28, s97
-; SI-NEXT: v_mov_b32_e32 v27, s96
-; SI-NEXT: v_mov_b32_e32 v26, s87
+; SI-NEXT: v_mov_b32_e32 v57, s50
+; SI-NEXT: v_mov_b32_e32 v56, s48
+; SI-NEXT: v_mov_b32_e32 v59, s39
+; SI-NEXT: v_mov_b32_e32 v58, s37
+; SI-NEXT: v_mov_b32_e32 v61, s36
+; SI-NEXT: v_mov_b32_e32 v60, s34
+; SI-NEXT: v_mov_b32_e32 v37, s31
+; SI-NEXT: v_mov_b32_e32 v36, s99
+; SI-NEXT: v_mov_b32_e32 v33, s98
+; SI-NEXT: v_mov_b32_e32 v32, s96
+; SI-NEXT: v_mov_b32_e32 v31, s87
+; SI-NEXT: v_mov_b32_e32 v30, s86
+; SI-NEXT: v_mov_b32_e32 v29, s85
+; SI-NEXT: v_mov_b32_e32 v28, s84
+; SI-NEXT: v_mov_b32_e32 v27, s83
+; SI-NEXT: v_mov_b32_e32 v26, s82
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s4
; SI-NEXT: .LBB103_6: ; %end
@@ -227226,444 +227506,460 @@ define inreg <64 x bfloat> @bitcast_v64i16_to_v64bf16_scalar(<64 x i16> inreg %a
; SI-NEXT: v_writelane_b32 v33, s99, 33
; SI-NEXT: v_writelane_b32 v33, s30, 34
; SI-NEXT: v_writelane_b32 v33, s31, 35
-; SI-NEXT: s_lshr_b32 s4, s16, 16
+; SI-NEXT: s_lshr_b32 s78, s16, 16
; SI-NEXT: ; implicit-def: $vgpr34 : SGPR spill to VGPR lane
-; SI-NEXT: s_lshr_b32 s46, s17, 16
+; SI-NEXT: s_lshr_b32 s77, s17, 16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v34, s4, 0
-; SI-NEXT: s_lshr_b32 s47, s18, 16
-; SI-NEXT: v_writelane_b32 v34, s46, 1
-; SI-NEXT: s_lshr_b32 s57, s19, 16
-; SI-NEXT: v_writelane_b32 v34, s47, 2
-; SI-NEXT: s_lshr_b32 s59, s20, 16
-; SI-NEXT: v_writelane_b32 v34, s57, 3
-; SI-NEXT: s_lshr_b32 s61, s21, 16
-; SI-NEXT: v_writelane_b32 v34, s59, 4
-; SI-NEXT: s_lshr_b32 s63, s22, 16
-; SI-NEXT: v_writelane_b32 v34, s61, 5
-; SI-NEXT: s_lshr_b32 s73, s23, 16
-; SI-NEXT: v_writelane_b32 v34, s63, 6
-; SI-NEXT: s_lshr_b32 s75, s24, 16
-; SI-NEXT: v_writelane_b32 v34, s73, 7
-; SI-NEXT: s_lshr_b32 s76, s25, 16
-; SI-NEXT: v_writelane_b32 v34, s75, 8
-; SI-NEXT: s_lshr_b32 s77, s26, 16
-; SI-NEXT: v_writelane_b32 v34, s76, 9
-; SI-NEXT: s_lshr_b32 s78, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s77, 10
-; SI-NEXT: s_lshr_b32 s79, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s78, 11
-; SI-NEXT: s_lshr_b32 s88, s29, 16
-; SI-NEXT: v_readfirstlane_b32 s54, v0
-; SI-NEXT: v_writelane_b32 v34, s79, 12
-; SI-NEXT: v_readfirstlane_b32 s64, v1
-; SI-NEXT: s_lshr_b32 s89, s54, 16
-; SI-NEXT: v_writelane_b32 v34, s88, 13
-; SI-NEXT: v_readfirstlane_b32 s66, v2
-; SI-NEXT: s_lshr_b32 s90, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s89, 14
-; SI-NEXT: v_readfirstlane_b32 s68, v3
-; SI-NEXT: s_lshr_b32 s91, s66, 16
-; SI-NEXT: v_writelane_b32 v34, s90, 15
-; SI-NEXT: v_readfirstlane_b32 s70, v4
-; SI-NEXT: s_lshr_b32 s92, s68, 16
-; SI-NEXT: v_writelane_b32 v34, s91, 16
-; SI-NEXT: v_readfirstlane_b32 s80, v5
-; SI-NEXT: s_lshr_b32 s93, s70, 16
-; SI-NEXT: v_writelane_b32 v34, s92, 17
-; SI-NEXT: v_readfirstlane_b32 s82, v6
-; SI-NEXT: s_lshr_b32 s94, s80, 16
-; SI-NEXT: v_writelane_b32 v34, s93, 18
-; SI-NEXT: v_readfirstlane_b32 s84, v7
-; SI-NEXT: s_lshr_b32 s95, s82, 16
-; SI-NEXT: v_writelane_b32 v34, s94, 19
-; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: v_readfirstlane_b32 s40, v16
-; SI-NEXT: v_readfirstlane_b32 s14, v15
-; SI-NEXT: v_readfirstlane_b32 s12, v14
-; SI-NEXT: v_readfirstlane_b32 s10, v13
-; SI-NEXT: v_readfirstlane_b32 s8, v12
-; SI-NEXT: v_readfirstlane_b32 s6, v11
-; SI-NEXT: v_readfirstlane_b32 s98, v10
-; SI-NEXT: v_readfirstlane_b32 s96, v9
-; SI-NEXT: v_readfirstlane_b32 s86, v8
-; SI-NEXT: s_lshr_b32 vcc_lo, s84, 16
-; SI-NEXT: v_writelane_b32 v34, s95, 20
-; SI-NEXT: s_lshr_b32 s41, s42, 16
-; SI-NEXT: s_lshr_b32 s15, s40, 16
-; SI-NEXT: s_lshr_b32 s13, s14, 16
-; SI-NEXT: s_lshr_b32 s11, s12, 16
-; SI-NEXT: s_lshr_b32 s9, s10, 16
-; SI-NEXT: s_lshr_b32 s7, s8, 16
-; SI-NEXT: s_lshr_b32 s99, s6, 16
-; SI-NEXT: s_lshr_b32 s97, s98, 16
-; SI-NEXT: s_lshr_b32 s30, s96, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_writelane_b32 v34, vcc_lo, 21
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_writelane_b32 v34, vcc_hi, 22
-; SI-NEXT: v_writelane_b32 v34, s30, 23
+; SI-NEXT: v_writelane_b32 v34, s78, 0
+; SI-NEXT: s_lshr_b32 s76, s18, 16
+; SI-NEXT: v_writelane_b32 v34, s77, 1
+; SI-NEXT: s_lshr_b32 s74, s19, 16
+; SI-NEXT: v_writelane_b32 v34, s76, 2
+; SI-NEXT: s_lshr_b32 s73, s20, 16
+; SI-NEXT: v_writelane_b32 v34, s74, 3
+; SI-NEXT: s_lshr_b32 s63, s21, 16
+; SI-NEXT: v_writelane_b32 v34, s73, 4
+; SI-NEXT: s_lshr_b32 s7, s22, 16
+; SI-NEXT: v_writelane_b32 v34, s63, 5
+; SI-NEXT: s_lshr_b32 s75, s23, 16
+; SI-NEXT: v_writelane_b32 v34, s7, 6
+; SI-NEXT: s_lshr_b32 s62, s24, 16
+; SI-NEXT: v_writelane_b32 v34, s75, 7
+; SI-NEXT: s_lshr_b32 s61, s25, 16
+; SI-NEXT: v_writelane_b32 v34, s62, 8
+; SI-NEXT: s_lshr_b32 s72, s26, 16
+; SI-NEXT: v_writelane_b32 v34, s61, 9
+; SI-NEXT: s_lshr_b32 s53, s27, 16
+; SI-NEXT: v_writelane_b32 v34, s72, 10
+; SI-NEXT: s_lshr_b32 s49, s28, 16
+; SI-NEXT: v_writelane_b32 v34, s53, 11
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: v_readfirstlane_b32 s9, v0
+; SI-NEXT: v_writelane_b32 v34, s49, 12
+; SI-NEXT: s_lshr_b32 s96, s9, 16
+; SI-NEXT: v_writelane_b32 v34, s56, 13
+; SI-NEXT: v_readfirstlane_b32 s11, v1
+; SI-NEXT: v_writelane_b32 v34, s96, 14
+; SI-NEXT: s_lshr_b32 s87, s11, 16
+; SI-NEXT: v_writelane_b32 v34, s9, 15
+; SI-NEXT: v_readfirstlane_b32 s6, v2
+; SI-NEXT: v_writelane_b32 v34, s87, 16
+; SI-NEXT: s_lshr_b32 s84, s6, 16
+; SI-NEXT: v_writelane_b32 v34, s11, 17
+; SI-NEXT: v_readfirstlane_b32 s14, v3
+; SI-NEXT: v_writelane_b32 v34, s84, 18
+; SI-NEXT: s_lshr_b32 s83, s14, 16
+; SI-NEXT: v_writelane_b32 v34, s6, 19
+; SI-NEXT: v_writelane_b32 v34, s83, 20
+; SI-NEXT: v_readfirstlane_b32 s40, v4
+; SI-NEXT: v_writelane_b32 v34, s14, 21
+; SI-NEXT: v_readfirstlane_b32 s42, v5
+; SI-NEXT: v_writelane_b32 v34, s40, 22
+; SI-NEXT: v_readfirstlane_b32 s44, v6
+; SI-NEXT: v_writelane_b32 v34, s42, 23
+; SI-NEXT: v_readfirstlane_b32 s31, v7
+; SI-NEXT: v_writelane_b32 v34, s44, 24
+; SI-NEXT: v_readfirstlane_b32 s37, v8
+; SI-NEXT: v_writelane_b32 v34, s31, 25
+; SI-NEXT: v_readfirstlane_b32 s50, v9
+; SI-NEXT: v_writelane_b32 v34, s37, 26
+; SI-NEXT: v_readfirstlane_b32 s52, v11
+; SI-NEXT: v_readfirstlane_b32 s55, v10
+; SI-NEXT: v_writelane_b32 v34, s50, 27
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: v_readfirstlane_b32 s60, v16
+; SI-NEXT: v_readfirstlane_b32 s34, v15
+; SI-NEXT: v_readfirstlane_b32 s57, v14
+; SI-NEXT: v_readfirstlane_b32 s47, v13
+; SI-NEXT: v_readfirstlane_b32 s45, v12
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_writelane_b32 v34, s55, 28
+; SI-NEXT: s_lshr_b32 s4, s5, 16
+; SI-NEXT: s_lshr_b32 s95, s60, 16
+; SI-NEXT: s_lshr_b32 vcc_lo, s34, 16
+; SI-NEXT: s_lshr_b32 s35, s57, 16
+; SI-NEXT: s_lshr_b32 s38, s47, 16
+; SI-NEXT: s_lshr_b32 s39, s45, 16
+; SI-NEXT: s_lshr_b32 s64, s55, 16
+; SI-NEXT: s_lshr_b32 s67, s50, 16
+; SI-NEXT: s_lshr_b32 s68, s37, 16
+; SI-NEXT: s_lshr_b32 s70, s31, 16
+; SI-NEXT: s_lshr_b32 s71, s44, 16
+; SI-NEXT: s_lshr_b32 s80, s42, 16
+; SI-NEXT: s_lshr_b32 s81, s40, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v18
+; SI-NEXT: v_writelane_b32 v34, s51, 29
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v34, s52, 30
+; SI-NEXT: v_writelane_b32 v34, s45, 31
; SI-NEXT: s_cbranch_scc0 .LBB107_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_lshl_b32 s44, s4, 16
-; SI-NEXT: s_lshl_b32 s4, s24, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 26
-; SI-NEXT: s_lshl_b32 s4, s25, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 27
-; SI-NEXT: s_lshl_b32 s4, s26, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 28
-; SI-NEXT: s_lshl_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 29
-; SI-NEXT: s_lshl_b32 s4, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 25
-; SI-NEXT: s_lshl_b32 s4, s79, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 30
-; SI-NEXT: s_lshl_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 24
-; SI-NEXT: s_lshl_b32 s4, s88, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 31
-; SI-NEXT: s_lshl_b32 s4, s89, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 32
-; SI-NEXT: s_lshl_b32 s4, s90, 16
-; SI-NEXT: s_lshl_b32 s43, s16, 16
-; SI-NEXT: s_lshl_b32 s45, s17, 16
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s56, s18, 16
-; SI-NEXT: s_lshl_b32 s47, s47, 16
-; SI-NEXT: s_lshl_b32 s58, s19, 16
-; SI-NEXT: s_lshl_b32 s57, s57, 16
-; SI-NEXT: s_lshl_b32 s60, s20, 16
-; SI-NEXT: s_lshl_b32 s59, s59, 16
-; SI-NEXT: s_lshl_b32 s62, s21, 16
-; SI-NEXT: s_lshl_b32 s61, s61, 16
-; SI-NEXT: s_lshl_b32 s72, s22, 16
-; SI-NEXT: s_lshl_b32 s63, s63, 16
-; SI-NEXT: s_lshl_b32 s74, s23, 16
-; SI-NEXT: s_lshl_b32 s73, s73, 16
+; SI-NEXT: s_lshl_b32 s8, s24, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 32
+; SI-NEXT: s_lshl_b32 s8, s25, 16
+; SI-NEXT: s_lshl_b32 s93, s16, 16
+; SI-NEXT: s_lshl_b32 s94, s78, 16
+; SI-NEXT: s_lshl_b32 s92, s17, 16
+; SI-NEXT: s_lshl_b32 s91, s77, 16
+; SI-NEXT: s_lshl_b32 s89, s18, 16
+; SI-NEXT: s_lshl_b32 s90, s76, 16
+; SI-NEXT: s_lshl_b32 s79, s19, 16
+; SI-NEXT: s_lshl_b32 s88, s74, 16
+; SI-NEXT: s_lshl_b32 s76, s20, 16
+; SI-NEXT: s_lshl_b32 s78, s73, 16
+; SI-NEXT: s_lshl_b32 s77, s21, 16
+; SI-NEXT: s_lshl_b32 s74, s63, 16
+; SI-NEXT: s_lshl_b32 s63, s22, 16
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s73, s23, 16
; SI-NEXT: s_lshl_b32 s75, s75, 16
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_lshl_b32 s77, s77, 16
-; SI-NEXT: s_lshl_b32 s78, s78, 16
-; SI-NEXT: s_lshl_b32 s83, s54, 16
-; SI-NEXT: s_lshl_b32 s71, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 33
-; SI-NEXT: s_lshl_b32 s81, s66, 16
-; SI-NEXT: s_lshl_b32 s87, s91, 16
-; SI-NEXT: s_lshl_b32 s55, s68, 16
-; SI-NEXT: s_lshl_b32 s85, s92, 16
-; SI-NEXT: s_lshl_b32 s52, s70, 16
-; SI-NEXT: s_lshl_b32 s67, s93, 16
-; SI-NEXT: s_lshl_b32 s50, s80, 16
-; SI-NEXT: s_lshl_b32 s69, s94, 16
-; SI-NEXT: s_lshl_b32 s48, s82, 16
-; SI-NEXT: s_lshl_b32 s65, s95, 16
-; SI-NEXT: s_lshl_b32 s38, s84, 16
-; SI-NEXT: s_lshl_b32 s53, vcc_lo, 16
-; SI-NEXT: s_lshl_b32 s36, s86, 16
-; SI-NEXT: s_lshl_b32 s51, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s34, s96, 16
-; SI-NEXT: s_lshl_b32 s49, s30, 16
-; SI-NEXT: s_lshl_b32 s30, s98, 16
-; SI-NEXT: s_lshl_b32 s39, s97, 16
-; SI-NEXT: s_lshl_b32 vcc_lo, s6, 16
-; SI-NEXT: s_lshl_b32 s37, s99, 16
-; SI-NEXT: s_lshl_b32 s94, s8, 16
-; SI-NEXT: s_lshl_b32 s35, s7, 16
-; SI-NEXT: s_lshl_b32 s92, s10, 16
-; SI-NEXT: s_lshl_b32 s31, s9, 16
-; SI-NEXT: s_lshl_b32 s95, s12, 16
-; SI-NEXT: s_lshl_b32 vcc_hi, s11, 16
-; SI-NEXT: s_lshl_b32 s89, s14, 16
-; SI-NEXT: s_lshl_b32 s93, s13, 16
-; SI-NEXT: s_lshl_b32 s90, s40, 16
-; SI-NEXT: s_lshl_b32 s91, s15, 16
-; SI-NEXT: s_lshl_b32 s79, s42, 16
-; SI-NEXT: s_lshl_b32 s88, s41, 16
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 33
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_lshl_b32 s46, s26, 16
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s66, s27, 16
+; SI-NEXT: s_lshl_b32 s99, s53, 16
+; SI-NEXT: s_lshl_b32 s54, s28, 16
+; SI-NEXT: s_lshl_b32 s69, s49, 16
+; SI-NEXT: s_lshl_b32 s36, s29, 16
+; SI-NEXT: s_lshl_b32 s65, s56, 16
+; SI-NEXT: s_lshl_b32 s43, s9, 16
+; SI-NEXT: s_lshl_b32 s48, s96, 16
+; SI-NEXT: s_lshl_b32 s15, s11, 16
+; SI-NEXT: s_lshl_b32 s30, s87, 16
+; SI-NEXT: s_lshl_b32 s13, s6, 16
+; SI-NEXT: s_lshl_b32 s41, s84, 16
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_lshl_b32 s6, s83, 16
+; SI-NEXT: s_lshl_b32 s11, s40, 16
+; SI-NEXT: s_mov_b32 s40, s81
+; SI-NEXT: s_lshl_b32 s12, s81, 16
+; SI-NEXT: s_lshl_b32 s96, s42, 16
+; SI-NEXT: s_mov_b32 s42, s80
+; SI-NEXT: s_lshl_b32 s10, s80, 16
+; SI-NEXT: s_lshl_b32 s84, s44, 16
+; SI-NEXT: s_mov_b32 s44, s71
+; SI-NEXT: s_lshl_b32 s9, s71, 16
+; SI-NEXT: s_lshl_b32 s83, s31, 16
+; SI-NEXT: s_mov_b32 s31, s70
+; SI-NEXT: s_lshl_b32 s8, s70, 16
+; SI-NEXT: s_lshl_b32 s80, s37, 16
+; SI-NEXT: s_mov_b32 s37, s68
+; SI-NEXT: s_lshl_b32 vcc_hi, s68, 16
+; SI-NEXT: s_lshl_b32 s71, s50, 16
+; SI-NEXT: s_mov_b32 s50, s67
+; SI-NEXT: s_lshl_b32 s87, s67, 16
+; SI-NEXT: s_lshl_b32 s68, s55, 16
+; SI-NEXT: s_mov_b32 s55, s64
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_lshl_b32 s67, s52, 16
+; SI-NEXT: s_lshl_b32 s81, s51, 16
+; SI-NEXT: s_lshl_b32 s64, s45, 16
+; SI-NEXT: s_mov_b32 s45, s39
+; SI-NEXT: s_lshl_b32 s70, s39, 16
+; SI-NEXT: s_mov_b32 s82, s47
+; SI-NEXT: s_lshl_b32 s52, s47, 16
+; SI-NEXT: s_mov_b32 s47, s38
+; SI-NEXT: s_lshl_b32 s53, s38, 16
+; SI-NEXT: s_mov_b32 s58, s57
+; SI-NEXT: s_lshl_b32 s49, s57, 16
+; SI-NEXT: s_mov_b32 s57, s35
+; SI-NEXT: s_lshl_b32 s51, s35, 16
+; SI-NEXT: s_mov_b32 s85, s34
+; SI-NEXT: s_lshl_b32 s38, s34, 16
+; SI-NEXT: s_mov_b32 s86, vcc_lo
+; SI-NEXT: s_lshl_b32 s39, vcc_lo, 16
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_lshl_b32 s34, s60, 16
+; SI-NEXT: s_mov_b32 s60, s95
+; SI-NEXT: s_lshl_b32 s35, s95, 16
+; SI-NEXT: s_mov_b32 s97, s5
+; SI-NEXT: s_lshl_b32 s95, s5, 16
+; SI-NEXT: s_mov_b32 s98, s4
+; SI-NEXT: s_lshl_b32 vcc_lo, s4, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB107_3
; SI-NEXT: .LBB107_2:
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: s_mov_b32 s97, s5
+; SI-NEXT: s_mov_b32 s98, s4
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_mov_b32 s60, s95
+; SI-NEXT: s_mov_b32 s85, s34
+; SI-NEXT: s_mov_b32 s86, vcc_lo
+; SI-NEXT: s_mov_b32 s58, s57
+; SI-NEXT: s_mov_b32 s57, s35
+; SI-NEXT: s_mov_b32 s82, s47
+; SI-NEXT: s_mov_b32 s47, s38
+; SI-NEXT: s_mov_b32 s45, s39
+; SI-NEXT: s_mov_b32 s55, s64
+; SI-NEXT: s_mov_b32 s50, s67
+; SI-NEXT: s_mov_b32 s37, s68
+; SI-NEXT: s_mov_b32 s31, s70
+; SI-NEXT: s_mov_b32 s44, s71
+; SI-NEXT: s_mov_b32 s42, s80
+; SI-NEXT: s_mov_b32 s40, s81
; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr93
+; SI-NEXT: ; implicit-def: $sgpr94
+; SI-NEXT: ; implicit-def: $sgpr92
+; SI-NEXT: ; implicit-def: $sgpr91
+; SI-NEXT: ; implicit-def: $sgpr89
+; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr75
+; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr77
; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr43
-; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr45
-; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr47
-; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr57
-; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $sgpr59
-; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr61
-; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr63
+; SI-NEXT: ; implicit-def: $sgpr77
; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: ; implicit-def: $sgpr63
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: ; implicit-def: $sgpr73
-; SI-NEXT: ; kill: killed $sgpr75
; SI-NEXT: ; implicit-def: $sgpr75
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; kill: killed $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; kill: killed $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; kill: killed $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr61
+; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: ; implicit-def: $sgpr66
+; SI-NEXT: ; implicit-def: $sgpr99
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr43
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr15
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr13
+; SI-NEXT: ; implicit-def: $sgpr41
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr11
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr96
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr84
+; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr83
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr81
; SI-NEXT: ; implicit-def: $sgpr87
-; SI-NEXT: ; implicit-def: $sgpr55
-; SI-NEXT: ; implicit-def: $sgpr85
-; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr68
+; SI-NEXT: ; implicit-def: $sgpr56
; SI-NEXT: ; implicit-def: $sgpr67
-; SI-NEXT: ; implicit-def: $sgpr50
-; SI-NEXT: ; implicit-def: $sgpr69
-; SI-NEXT: ; implicit-def: $sgpr48
-; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr52
; SI-NEXT: ; implicit-def: $sgpr53
-; SI-NEXT: ; implicit-def: $sgpr36
-; SI-NEXT: ; implicit-def: $sgpr51
-; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr49
-; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr38
; SI-NEXT: ; implicit-def: $sgpr39
-; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr37
-; SI-NEXT: ; implicit-def: $sgpr94
+; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr35
-; SI-NEXT: ; implicit-def: $sgpr92
-; SI-NEXT: ; implicit-def: $sgpr31
; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $vcc_hi
-; SI-NEXT: ; implicit-def: $sgpr89
-; SI-NEXT: ; implicit-def: $sgpr93
-; SI-NEXT: ; implicit-def: $sgpr90
-; SI-NEXT: ; implicit-def: $sgpr91
-; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr79
+; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: .LBB107_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s4, s79
-; SI-NEXT: s_mov_b32 s5, s90
-; SI-NEXT: s_mov_b32 s79, s89
-; SI-NEXT: s_mov_b32 s89, s95
-; SI-NEXT: s_mov_b32 s90, s92
-; SI-NEXT: s_mov_b32 s92, s94
-; SI-NEXT: s_mov_b32 s94, vcc_lo
+; SI-NEXT: s_mov_b32 s4, s95
+; SI-NEXT: s_mov_b32 s5, vcc_lo
; SI-NEXT: s_mov_b32 s95, vcc_hi
-; SI-NEXT: s_mov_b32 vcc_lo, s30
-; SI-NEXT: s_mov_b32 vcc_hi, s31
-; SI-NEXT: s_mov_b32 s30, s34
-; SI-NEXT: s_mov_b32 s31, s35
-; SI-NEXT: s_mov_b32 s34, s36
-; SI-NEXT: s_mov_b32 s35, s37
-; SI-NEXT: s_mov_b32 s36, s38
-; SI-NEXT: s_mov_b32 s37, s39
-; SI-NEXT: s_mov_b32 s38, s48
-; SI-NEXT: s_mov_b32 s39, s49
-; SI-NEXT: s_mov_b32 s48, s50
-; SI-NEXT: s_mov_b32 s49, s51
-; SI-NEXT: s_mov_b32 s50, s52
-; SI-NEXT: s_mov_b32 s51, s53
-; SI-NEXT: s_mov_b32 s52, s55
-; SI-NEXT: s_mov_b32 s53, s65
-; SI-NEXT: s_mov_b32 s55, s81
-; SI-NEXT: s_mov_b32 s65, s69
-; SI-NEXT: s_mov_b32 s69, s85
-; SI-NEXT: v_readlane_b32 s81, v34, 24
-; SI-NEXT: v_readlane_b32 s85, v34, 25
+; SI-NEXT: s_mov_b32 vcc_lo, s8
+; SI-NEXT: s_mov_b32 s8, s9
+; SI-NEXT: s_mov_b32 s9, s11
+; SI-NEXT: s_mov_b32 vcc_hi, s46
; SI-NEXT: s_cbranch_scc1 .LBB107_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
+; SI-NEXT: s_add_i32 s4, s97, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s5, s59, 3
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s7, s60, 16
+; SI-NEXT: s_or_b32 s5, s7, s5
+; SI-NEXT: s_add_i32 s7, s85, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s86, 16
+; SI-NEXT: s_or_b32 s59, s56, s7
+; SI-NEXT: s_add_i32 s7, s58, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s57, 16
+; SI-NEXT: s_or_b32 s57, s56, s7
+; SI-NEXT: s_add_i32 s7, s82, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s7, s47, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 31
+; SI-NEXT: s_add_i32 s47, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s7, s45, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 30
+; SI-NEXT: s_add_i32 s45, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 29
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s6, 16
+; SI-NEXT: s_or_b32 s7, s46, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 28
+; SI-NEXT: s_add_i32 s46, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s7, s56, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 27
+; SI-NEXT: s_add_i32 s58, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s7, s56, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 26
+; SI-NEXT: s_add_i32 s56, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s37, 16
+; SI-NEXT: s_or_b32 s7, s60, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 25
+; SI-NEXT: s_add_i32 s60, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s31, 16
+; SI-NEXT: s_or_b32 s7, s61, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 24
+; SI-NEXT: s_add_i32 s95, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s7, s44, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 23
+; SI-NEXT: s_add_i32 s44, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s7, s42, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 22
+; SI-NEXT: s_add_i32 s42, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s7, s40, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 21
+; SI-NEXT: s_add_i32 s40, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 20
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s14, s6, 16
+; SI-NEXT: s_or_b32 s7, s14, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 19
+; SI-NEXT: s_add_i32 s14, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 18
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s13, s6, 0x30000
+; SI-NEXT: v_readlane_b32 s6, v34, 17
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: v_readlane_b32 s7, v34, 16
+; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
+; SI-NEXT: s_add_i32 s11, s6, 0x30000
+; SI-NEXT: v_readlane_b32 s6, v34, 15
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_add_i32 s10, s10, 3
-; SI-NEXT: s_or_b32 s7, s7, s8
+; SI-NEXT: v_readlane_b32 s7, v34, 14
; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s99, 16
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s98, s98, 3
-; SI-NEXT: s_and_b32 s12, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_or_b32 s9, s9, s10
-; SI-NEXT: s_add_i32 s8, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s98, 0xffff
-; SI-NEXT: s_lshl_b32 s10, s97, 16
-; SI-NEXT: s_add_i32 s14, s14, 3
-; SI-NEXT: s_or_b32 s11, s11, s12
-; SI-NEXT: s_or_b32 s6, s10, s6
-; SI-NEXT: s_add_i32 s96, s96, 3
-; SI-NEXT: v_readlane_b32 s12, v34, 23
-; SI-NEXT: s_add_i32 s42, s42, 3
-; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s13, 16
-; SI-NEXT: s_add_i32 s10, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s4, s42, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s41, 16
-; SI-NEXT: s_add_i32 s40, s40, 3
-; SI-NEXT: s_or_b32 s13, s13, s14
-; SI-NEXT: s_or_b32 s6, s12, s6
-; SI-NEXT: s_add_i32 s86, s86, 3
-; SI-NEXT: v_readlane_b32 s14, v34, 22
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s5, s40, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s15, 16
-; SI-NEXT: s_add_i32 s12, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s14, s14, 16
-; SI-NEXT: s_or_b32 s5, s15, s5
-; SI-NEXT: s_or_b32 s6, s14, s6
-; SI-NEXT: s_add_i32 s84, s84, 3
-; SI-NEXT: v_readlane_b32 s15, v34, 21
-; SI-NEXT: s_add_i32 s14, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s84, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s15, 16
-; SI-NEXT: s_or_b32 s6, s15, s6
-; SI-NEXT: s_add_i32 s82, s82, 3
-; SI-NEXT: v_readlane_b32 s40, v34, 20
-; SI-NEXT: s_add_i32 s15, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s82, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s40, 16
-; SI-NEXT: s_or_b32 s6, s40, s6
-; SI-NEXT: s_add_i32 s80, s80, 3
-; SI-NEXT: v_readlane_b32 s41, v34, 19
-; SI-NEXT: s_add_i32 s40, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s41, 16
-; SI-NEXT: s_or_b32 s6, s41, s6
-; SI-NEXT: s_add_i32 s70, s70, 3
-; SI-NEXT: v_readlane_b32 s42, v34, 18
-; SI-NEXT: s_add_i32 s41, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s70, 0xffff
-; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_or_b32 s6, s42, s6
-; SI-NEXT: s_add_i32 s68, s68, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 17
-; SI-NEXT: s_add_i32 s42, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
-; SI-NEXT: s_add_i32 s66, s66, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 16
-; SI-NEXT: s_add_i32 s67, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s66, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
-; SI-NEXT: s_add_i32 s64, s64, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 15
-; SI-NEXT: s_add_i32 s65, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s64, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
-; SI-NEXT: s_add_i32 s54, s54, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 14
-; SI-NEXT: s_add_i32 s88, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s54, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_add_i32 s90, s6, 0x30000
+; SI-NEXT: v_readlane_b32 s7, v34, 13
+; SI-NEXT: s_add_i32 s9, s6, 0x30000
; SI-NEXT: s_and_b32 s6, s29, 0xffff
-; SI-NEXT: v_readlane_b32 s29, v34, 13
-; SI-NEXT: s_lshl_b32 s29, s29, 16
-; SI-NEXT: s_or_b32 s6, s29, s6
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
+; SI-NEXT: s_add_i32 s8, s6, 0x30000
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s29, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s28, 0xffff
-; SI-NEXT: v_readlane_b32 s28, v34, 12
-; SI-NEXT: s_lshl_b32 s28, s28, 16
-; SI-NEXT: s_or_b32 s6, s28, s6
+; SI-NEXT: v_readlane_b32 s6, v34, 12
+; SI-NEXT: s_and_b32 s7, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s10, s6, 0x30000
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_add_i32 s28, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s27, 0xffff
-; SI-NEXT: v_readlane_b32 s27, v34, 11
-; SI-NEXT: s_lshl_b32 s27, s27, 16
-; SI-NEXT: s_or_b32 s6, s27, s6
+; SI-NEXT: v_readlane_b32 s6, v34, 11
+; SI-NEXT: s_and_b32 s7, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s12, s6, 0x30000
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_add_i32 s27, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s26, 0xffff
-; SI-NEXT: v_readlane_b32 s26, v34, 10
-; SI-NEXT: s_lshl_b32 s26, s26, 16
-; SI-NEXT: s_or_b32 s6, s26, s6
+; SI-NEXT: v_readlane_b32 s6, v34, 10
+; SI-NEXT: s_and_b32 s7, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s15, s6, 0x30000
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_add_i32 s26, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s25, 0xffff
-; SI-NEXT: v_readlane_b32 s25, v34, 9
-; SI-NEXT: s_lshl_b32 s25, s25, 16
-; SI-NEXT: s_or_b32 s6, s25, s6
-; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 9
+; SI-NEXT: s_and_b32 s7, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s25, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s24, 0xffff
-; SI-NEXT: v_readlane_b32 s24, v34, 8
-; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: s_or_b32 s6, s24, s6
-; SI-NEXT: s_add_i32 s23, s23, 3
+; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 8
+; SI-NEXT: s_and_b32 s7, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s24, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s23, 0xffff
-; SI-NEXT: v_readlane_b32 s23, v34, 7
-; SI-NEXT: s_lshl_b32 s23, s23, 16
-; SI-NEXT: s_or_b32 s6, s23, s6
-; SI-NEXT: s_add_i32 s22, s22, 3
+; SI-NEXT: s_add_i32 s23, s23, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 7
+; SI-NEXT: s_and_b32 s7, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s23, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s22, 0xffff
-; SI-NEXT: v_readlane_b32 s22, v34, 6
-; SI-NEXT: s_lshl_b32 s22, s22, 16
-; SI-NEXT: s_or_b32 s6, s22, s6
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s22, s22, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 6
+; SI-NEXT: s_and_b32 s7, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s22, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s21, 0xffff
-; SI-NEXT: v_readlane_b32 s21, v34, 5
-; SI-NEXT: s_lshl_b32 s21, s21, 16
-; SI-NEXT: s_or_b32 s6, s21, s6
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 5
+; SI-NEXT: s_and_b32 s7, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s7, s6, 0x30000
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s21, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s20, 0xffff
-; SI-NEXT: v_readlane_b32 s20, v34, 4
-; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_or_b32 s6, s20, s6
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 4
+; SI-NEXT: s_and_b32 s20, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s20
; SI-NEXT: s_add_i32 s20, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s19, 0xffff
-; SI-NEXT: v_readlane_b32 s19, v34, 3
-; SI-NEXT: s_lshl_b32 s19, s19, 16
-; SI-NEXT: s_or_b32 s6, s19, s6
-; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 3
+; SI-NEXT: s_and_b32 s19, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s19
; SI-NEXT: s_add_i32 s19, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s18, 0xffff
-; SI-NEXT: v_readlane_b32 s18, v34, 2
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_or_b32 s6, s18, s6
-; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 2
+; SI-NEXT: s_and_b32 s18, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s18
; SI-NEXT: s_add_i32 s18, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s17, 0xffff
-; SI-NEXT: v_readlane_b32 s17, v34, 1
-; SI-NEXT: s_lshl_b32 s17, s17, 16
-; SI-NEXT: s_or_b32 s6, s17, s6
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 1
+; SI-NEXT: s_and_b32 s17, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s17
; SI-NEXT: s_add_i32 s17, s6, 0x30000
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: v_readlane_b32 s6, v34, 0
@@ -227671,218 +227967,204 @@ define inreg <64 x bfloat> @bitcast_v64i16_to_v64bf16_scalar(<64 x i16> inreg %a
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_or_b32 s6, s6, s16
; SI-NEXT: s_add_i32 s6, s6, 0x30000
-; SI-NEXT: s_and_b32 s44, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s6, 16
+; SI-NEXT: s_and_b32 s94, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s6, 16
; SI-NEXT: s_lshl_b32 s6, s24, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 26
-; SI-NEXT: s_lshl_b32 s6, s25, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 27
-; SI-NEXT: s_lshl_b32 s6, s26, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 28
-; SI-NEXT: s_lshl_b32 s6, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 29
-; SI-NEXT: s_and_b32 s6, s28, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s6, 30
-; SI-NEXT: s_and_b32 s6, s29, 0xffff0000
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_add_i32 s13, s13, 0x30000
-; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_add_i32 s9, s9, 0x30000
-; SI-NEXT: s_add_i32 s7, s7, 0x30000
-; SI-NEXT: v_writelane_b32 v34, s6, 31
-; SI-NEXT: s_and_b32 s6, s90, 0xffff0000
-; SI-NEXT: s_and_b32 s46, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s17, 16
-; SI-NEXT: s_and_b32 s47, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s56, s18, 16
-; SI-NEXT: s_and_b32 s57, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s58, s19, 16
-; SI-NEXT: s_and_b32 s59, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s60, s20, 16
-; SI-NEXT: s_and_b32 s61, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s62, s21, 16
-; SI-NEXT: s_and_b32 s63, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s72, s22, 16
-; SI-NEXT: s_and_b32 s73, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s74, s23, 16
-; SI-NEXT: s_and_b32 s75, s24, 0xffff0000
-; SI-NEXT: s_and_b32 s76, s25, 0xffff0000
-; SI-NEXT: s_and_b32 s77, s26, 0xffff0000
-; SI-NEXT: s_and_b32 s78, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s85, s28, 16
-; SI-NEXT: s_lshl_b32 s81, s29, 16
+; SI-NEXT: s_add_i32 s59, s59, 0x30000
+; SI-NEXT: s_add_i32 s57, s57, 0x30000
; SI-NEXT: v_writelane_b32 v34, s6, 32
-; SI-NEXT: s_lshl_b32 s83, s90, 16
-; SI-NEXT: s_and_b32 s6, s88, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s88, 16
-; SI-NEXT: s_and_b32 s87, s65, 0xffff0000
-; SI-NEXT: s_lshl_b32 s55, s65, 16
-; SI-NEXT: s_and_b32 s69, s67, 0xffff0000
-; SI-NEXT: s_lshl_b32 s52, s67, 16
-; SI-NEXT: s_and_b32 s67, s42, 0xffff0000
-; SI-NEXT: s_lshl_b32 s50, s42, 16
-; SI-NEXT: s_and_b32 s65, s41, 0xffff0000
-; SI-NEXT: s_lshl_b32 s48, s41, 16
-; SI-NEXT: s_and_b32 s53, s40, 0xffff0000
-; SI-NEXT: s_lshl_b32 s38, s40, 16
-; SI-NEXT: s_and_b32 s51, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s36, s15, 16
-; SI-NEXT: s_and_b32 s49, s14, 0xffff0000
-; SI-NEXT: s_lshl_b32 s34, s14, 16
-; SI-NEXT: s_and_b32 s39, s12, 0xffff0000
-; SI-NEXT: s_lshl_b32 s30, s12, 16
-; SI-NEXT: s_and_b32 s37, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s10, 16
-; SI-NEXT: s_and_b32 s35, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s8, 16
-; SI-NEXT: s_and_b32 s31, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s92, s7, 16
-; SI-NEXT: s_and_b32 vcc_hi, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s9, 16
-; SI-NEXT: s_and_b32 s95, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s89, s11, 16
-; SI-NEXT: s_and_b32 s93, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s79, s13, 16
-; SI-NEXT: s_and_b32 s91, s5, 0xffff0000
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_and_b32 s88, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s25, 16
+; SI-NEXT: s_and_b32 s91, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s92, s17, 16
+; SI-NEXT: s_and_b32 s90, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s18, 16
+; SI-NEXT: s_and_b32 s88, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s19, 16
+; SI-NEXT: s_and_b32 s78, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s76, s20, 16
+; SI-NEXT: s_and_b32 s74, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s7, 16
+; SI-NEXT: s_and_b32 s7, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s22, 16
+; SI-NEXT: s_and_b32 s75, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s23, 16
+; SI-NEXT: s_and_b32 s62, s24, 0xffff0000
+; SI-NEXT: s_and_b32 s61, s25, 0xffff0000
; SI-NEXT: v_writelane_b32 v34, s6, 33
+; SI-NEXT: s_and_b32 s72, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s15, 16
+; SI-NEXT: s_and_b32 s99, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s66, s12, 16
+; SI-NEXT: s_and_b32 s69, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s10, 16
+; SI-NEXT: s_and_b32 s65, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s8, 16
+; SI-NEXT: s_and_b32 s48, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s43, s9, 16
+; SI-NEXT: s_and_b32 s30, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s11, 16
+; SI-NEXT: s_and_b32 s41, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_and_b32 s6, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_and_b32 s12, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s40, 16
+; SI-NEXT: s_and_b32 s10, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s96, s42, 16
+; SI-NEXT: s_and_b32 s8, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s84, s44, 16
+; SI-NEXT: s_and_b32 vcc_lo, s95, 0xffff0000
+; SI-NEXT: s_lshl_b32 s83, s95, 16
+; SI-NEXT: s_and_b32 s95, s60, 0xffff0000
+; SI-NEXT: s_lshl_b32 s80, s60, 16
+; SI-NEXT: s_and_b32 s87, s56, 0xffff0000
+; SI-NEXT: s_lshl_b32 s71, s56, 16
+; SI-NEXT: s_and_b32 s56, s58, 0xffff0000
+; SI-NEXT: s_lshl_b32 s68, s58, 16
+; SI-NEXT: s_and_b32 s81, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s67, s46, 16
+; SI-NEXT: s_and_b32 s70, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s45, 16
+; SI-NEXT: s_and_b32 s53, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s47, 16
+; SI-NEXT: s_and_b32 s51, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s49, s57, 16
+; SI-NEXT: s_and_b32 s39, s59, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s59, 16
+; SI-NEXT: s_and_b32 s35, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s5, 16
+; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: .LBB107_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s93
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s91
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s92
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s90
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s89
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s79
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s76
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s74
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s77
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s7
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s63
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s75
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s73
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s75
-; SI-NEXT: v_readlane_b32 s6, v34, 26
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s62
+; SI-NEXT: v_readlane_b32 s7, v34, 32
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s7
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s76
-; SI-NEXT: v_readlane_b32 s6, v34, 27
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s61
+; SI-NEXT: v_readlane_b32 s7, v34, 33
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s7
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s77
-; SI-NEXT: v_readlane_b32 s6, v34, 28
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s72
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, vcc_hi
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s78
-; SI-NEXT: v_readlane_b32 s6, v34, 29
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s99
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s66
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 30
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s69
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s54
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 31
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s36
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 32
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s48
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s43
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 33
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s30
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s15
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s55
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s13
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s6
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s52
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s14
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s12
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s50
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s9
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s10
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s48
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s96
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s53
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s8
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s38
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s84
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s51
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, vcc_lo
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s36
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s83
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s49
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s95
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s34
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s80
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s39
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s87
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s30
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s71
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s37
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s56
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, vcc_lo
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s68
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s35
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s81
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s67
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s31
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s70
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s64
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, vcc_hi
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s52
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s49
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s38
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s5
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s34
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
; SI-NEXT: v_readlane_b32 s30, v33, 34
@@ -231605,24 +231887,24 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s99, 33
; SI-NEXT: v_writelane_b32 v32, s30, 34
; SI-NEXT: v_writelane_b32 v32, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s68, v0
-; SI-NEXT: v_readfirstlane_b32 s69, v2
-; SI-NEXT: s_lshr_b32 s14, s68, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v0
+; SI-NEXT: v_readfirstlane_b32 s34, v2
+; SI-NEXT: s_lshr_b32 s14, s30, 16
; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s71, v4
-; SI-NEXT: s_lshr_b32 s12, s69, 16
+; SI-NEXT: v_readfirstlane_b32 s36, v4
+; SI-NEXT: s_lshr_b32 s12, s34, 16
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_readfirstlane_b32 s80, v6
-; SI-NEXT: s_lshr_b32 s10, s71, 16
+; SI-NEXT: v_readfirstlane_b32 s38, v6
+; SI-NEXT: s_lshr_b32 s10, s36, 16
; SI-NEXT: v_writelane_b32 v33, s12, 1
-; SI-NEXT: v_readfirstlane_b32 s82, v8
-; SI-NEXT: s_lshr_b32 s8, s80, 16
+; SI-NEXT: v_readfirstlane_b32 s48, v8
+; SI-NEXT: s_lshr_b32 s8, s38, 16
; SI-NEXT: v_writelane_b32 v33, s10, 2
-; SI-NEXT: v_readfirstlane_b32 s84, v10
-; SI-NEXT: s_lshr_b32 s6, s82, 16
+; SI-NEXT: v_readfirstlane_b32 s50, v10
+; SI-NEXT: s_lshr_b32 s6, s48, 16
; SI-NEXT: v_writelane_b32 v33, s8, 3
-; SI-NEXT: s_lshr_b32 s4, s84, 16
+; SI-NEXT: s_lshr_b32 s4, s50, 16
; SI-NEXT: v_writelane_b32 v33, s6, 4
; SI-NEXT: v_readfirstlane_b32 s15, v1
; SI-NEXT: v_writelane_b32 v33, s4, 5
@@ -231637,36 +231919,36 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s5, v11
; SI-NEXT: v_writelane_b32 v33, s7, 10
; SI-NEXT: v_readfirstlane_b32 s75, v13
-; SI-NEXT: v_readfirstlane_b32 s86, v12
+; SI-NEXT: v_readfirstlane_b32 s87, v12
; SI-NEXT: v_writelane_b32 v33, s5, 11
-; SI-NEXT: s_lshr_b32 s74, s86, 16
+; SI-NEXT: s_lshr_b32 s74, s87, 16
; SI-NEXT: v_writelane_b32 v33, s75, 12
-; SI-NEXT: v_readfirstlane_b32 s72, v15
-; SI-NEXT: v_readfirstlane_b32 s96, v14
+; SI-NEXT: v_readfirstlane_b32 s73, v15
; SI-NEXT: v_writelane_b32 v33, s74, 13
-; SI-NEXT: s_lshr_b32 s73, s96, 16
-; SI-NEXT: v_writelane_b32 v33, s72, 14
+; SI-NEXT: v_readfirstlane_b32 s67, v14
+; SI-NEXT: v_writelane_b32 v33, s73, 14
+; SI-NEXT: s_lshr_b32 s72, s67, 16
+; SI-NEXT: v_writelane_b32 v33, s30, 15
; SI-NEXT: v_readfirstlane_b32 s62, v17
-; SI-NEXT: v_readfirstlane_b32 s98, v16
-; SI-NEXT: v_writelane_b32 v33, s73, 15
-; SI-NEXT: s_mov_b32 s76, s16
+; SI-NEXT: v_readfirstlane_b32 s66, v16
+; SI-NEXT: v_writelane_b32 v33, s72, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s99, s28, 16
+; SI-NEXT: s_lshr_b32 s82, s28, 16
; SI-NEXT: s_lshr_b32 s95, s27, 16
-; SI-NEXT: s_lshr_b32 s97, s26, 16
+; SI-NEXT: s_lshr_b32 s81, s26, 16
; SI-NEXT: s_lshr_b32 s93, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
+; SI-NEXT: s_lshr_b32 s80, s24, 16
; SI-NEXT: s_lshr_b32 s91, s23, 16
-; SI-NEXT: s_lshr_b32 s85, s22, 16
+; SI-NEXT: s_lshr_b32 s71, s22, 16
; SI-NEXT: s_lshr_b32 s89, s21, 16
-; SI-NEXT: s_lshr_b32 s83, s20, 16
+; SI-NEXT: s_lshr_b32 s70, s20, 16
; SI-NEXT: s_lshr_b32 s79, s19, 16
-; SI-NEXT: s_lshr_b32 s81, s18, 16
+; SI-NEXT: s_lshr_b32 s69, s18, 16
; SI-NEXT: s_lshr_b32 s77, s17, 16
-; SI-NEXT: s_lshr_b32 s70, s16, 16
+; SI-NEXT: s_lshr_b32 s68, s16, 16
; SI-NEXT: s_lshr_b32 s65, s62, 16
-; SI-NEXT: s_lshr_b32 s63, s98, 16
-; SI-NEXT: s_lshr_b32 s55, s72, 16
+; SI-NEXT: s_lshr_b32 s63, s66, 16
+; SI-NEXT: s_lshr_b32 s55, s73, 16
; SI-NEXT: s_lshr_b32 s53, s75, 16
; SI-NEXT: s_lshr_b32 s51, s5, 16
; SI-NEXT: s_lshr_b32 s49, s7, 16
@@ -231674,152 +231956,153 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s37, s11, 16
; SI-NEXT: s_lshr_b32 s35, s13, 16
; SI-NEXT: s_lshr_b32 s31, s15, 16
-; SI-NEXT: v_readfirstlane_b32 s16, v18
-; SI-NEXT: v_writelane_b32 v33, s62, 16
-; SI-NEXT: s_cmp_lg_u32 s16, 0
-; SI-NEXT: v_writelane_b32 v33, s17, 17
-; SI-NEXT: v_writelane_b32 v33, s76, 18
+; SI-NEXT: v_readfirstlane_b32 s40, v18
+; SI-NEXT: v_writelane_b32 v33, s62, 17
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s34, 18
+; SI-NEXT: v_writelane_b32 v33, s63, 19
; SI-NEXT: s_cbranch_scc0 .LBB111_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s16, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s77, 16
-; SI-NEXT: s_or_b32 s61, s16, s17
-; SI-NEXT: s_and_b32 s16, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s79, 16
-; SI-NEXT: s_or_b32 s59, s16, s17
-; SI-NEXT: s_and_b32 s16, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s89, 16
-; SI-NEXT: s_or_b32 s57, s16, s17
-; SI-NEXT: s_and_b32 s16, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s91, 16
-; SI-NEXT: s_or_b32 s47, s16, s17
-; SI-NEXT: s_and_b32 s16, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s93, 16
-; SI-NEXT: s_or_b32 s45, s16, s17
-; SI-NEXT: s_and_b32 s16, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s95, 16
-; SI-NEXT: s_or_b32 s43, s16, s17
-; SI-NEXT: s_and_b32 s16, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s17, vcc_hi, 16
-; SI-NEXT: s_or_b32 s41, s16, s17
-; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s31, 16
-; SI-NEXT: s_or_b32 s15, s15, s16
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s35, 16
-; SI-NEXT: s_or_b32 s13, s13, s16
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s37, 16
-; SI-NEXT: s_or_b32 s11, s11, s16
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s39, 16
-; SI-NEXT: s_or_b32 s9, s9, s16
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s49, 16
-; SI-NEXT: s_or_b32 s7, s7, s16
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s51, 16
-; SI-NEXT: s_or_b32 s5, s5, s16
-; SI-NEXT: s_and_b32 s16, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s53, 16
-; SI-NEXT: s_mov_b32 s66, s53
-; SI-NEXT: s_or_b32 s53, s16, s17
-; SI-NEXT: s_and_b32 s16, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s55, 16
-; SI-NEXT: s_mov_b32 s67, s55
-; SI-NEXT: s_or_b32 s55, s16, s17
-; SI-NEXT: s_and_b32 s16, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s62, s65, 16
-; SI-NEXT: s_lshl_b32 s60, s70, 16
-; SI-NEXT: s_mov_b32 s17, s65
-; SI-NEXT: s_or_b32 s65, s16, s62
-; SI-NEXT: s_and_b32 s62, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s58, s81, 16
+; SI-NEXT: s_and_b32 s41, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s77, 16
+; SI-NEXT: s_or_b32 s61, s41, s43
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s79, 16
+; SI-NEXT: s_lshl_b32 s54, s72, 16
+; SI-NEXT: s_and_b32 s62, s62, 0xffff
+; SI-NEXT: s_lshl_b32 s72, s65, 16
+; SI-NEXT: s_lshl_b32 s60, s68, 16
+; SI-NEXT: s_or_b32 s59, s41, s43
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s89, 16
+; SI-NEXT: s_mov_b32 s97, s65
+; SI-NEXT: s_or_b32 s65, s62, s72
+; SI-NEXT: s_and_b32 s62, s16, 0xffff
+; SI-NEXT: s_or_b32 s57, s41, s43
+; SI-NEXT: s_and_b32 s41, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s91, 16
+; SI-NEXT: s_lshl_b32 s64, s63, 16
; SI-NEXT: s_or_b32 s62, s62, s60
-; SI-NEXT: s_mov_b32 s72, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[60:61], 16
-; SI-NEXT: s_and_b32 s60, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s83, 16
+; SI-NEXT: s_mov_b32 s63, s61
+; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
+; SI-NEXT: s_or_b32 s47, s41, s43
+; SI-NEXT: s_and_b32 s41, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: v_writelane_b32 v33, s60, 20
+; SI-NEXT: s_lshl_b32 s58, s69, 16
+; SI-NEXT: s_or_b32 s45, s41, s43
+; SI-NEXT: s_and_b32 s41, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s95, 16
; SI-NEXT: s_lshl_b32 s52, s74, 16
+; SI-NEXT: s_and_b32 s73, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s74, s55, 16
+; SI-NEXT: v_writelane_b32 v33, s61, 21
+; SI-NEXT: s_and_b32 s60, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s70, 16
+; SI-NEXT: s_or_b32 s43, s41, s43
+; SI-NEXT: s_and_b32 s41, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s76, vcc_hi, 16
+; SI-NEXT: s_mov_b32 s96, s55
+; SI-NEXT: s_or_b32 s55, s73, s74
; SI-NEXT: s_or_b32 s74, s60, s58
-; SI-NEXT: s_mov_b32 s60, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[58:59], 16
+; SI-NEXT: s_lshr_b64 s[98:99], s[58:59], 16
; SI-NEXT: s_and_b32 s58, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s46, s85, 16
-; SI-NEXT: s_mov_b32 s77, s72
+; SI-NEXT: s_lshl_b32 s46, s71, 16
+; SI-NEXT: s_or_b32 s41, s41, s76
+; SI-NEXT: s_and_b32 s15, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s31, 16
; SI-NEXT: s_or_b32 s72, s58, s56
; SI-NEXT: s_mov_b32 s58, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[56:57], 16
; SI-NEXT: s_and_b32 s56, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s87, 16
-; SI-NEXT: s_mov_b32 s79, s60
+; SI-NEXT: s_lshl_b32 s44, s80, 16
+; SI-NEXT: s_or_b32 s15, s15, s76
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s35, 16
; SI-NEXT: s_or_b32 s60, s56, s46
; SI-NEXT: s_mov_b32 s56, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[46:47], 16
; SI-NEXT: s_and_b32 s46, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s42, s97, 16
+; SI-NEXT: s_lshl_b32 s42, s81, 16
+; SI-NEXT: s_or_b32 s13, s13, s76
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s37, 16
; SI-NEXT: s_mov_b32 s89, s58
; SI-NEXT: s_or_b32 s58, s46, s44
; SI-NEXT: s_mov_b32 s46, s93
; SI-NEXT: s_lshr_b64 s[92:93], s[44:45], 16
; SI-NEXT: s_and_b32 s44, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s99, 16
+; SI-NEXT: s_lshl_b32 s40, s82, 16
+; SI-NEXT: s_or_b32 s11, s11, s76
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s39, 16
; SI-NEXT: s_mov_b32 s91, s56
; SI-NEXT: s_or_b32 s56, s44, s42
; SI-NEXT: s_mov_b32 s44, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[42:43], 16
; SI-NEXT: s_and_b32 s42, s28, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s9, s9, s76
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s49, 16
; SI-NEXT: s_mov_b32 s93, s46
; SI-NEXT: s_or_b32 s46, s42, s40
; SI-NEXT: s_mov_b32 s42, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[40:41], 16
-; SI-NEXT: s_and_b32 s40, s68, 0xffff
+; SI-NEXT: s_and_b32 s40, s30, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s7, s7, s76
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s51, 16
; SI-NEXT: s_mov_b32 s95, s44
; SI-NEXT: s_or_b32 s44, s40, s14
; SI-NEXT: s_mov_b32 s40, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[14:15], 16
-; SI-NEXT: s_and_b32 s14, s69, 0xffff
+; SI-NEXT: s_and_b32 s14, s34, 0xffff
; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s5, s5, s76
+; SI-NEXT: s_and_b32 s75, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s53, 16
; SI-NEXT: s_mov_b32 vcc_hi, s42
; SI-NEXT: s_or_b32 s42, s14, s12
; SI-NEXT: s_mov_b32 s14, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[12:13], 16
-; SI-NEXT: s_and_b32 s12, s71, 0xffff
+; SI-NEXT: s_and_b32 s12, s36, 0xffff
; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_mov_b32 s86, s53
+; SI-NEXT: s_or_b32 s53, s75, s76
; SI-NEXT: s_mov_b32 s31, s40
+; SI-NEXT: s_mov_b32 s76, s36
; SI-NEXT: s_or_b32 s40, s12, s10
; SI-NEXT: s_mov_b32 s12, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[10:11], 16
-; SI-NEXT: s_and_b32 s10, s80, 0xffff
+; SI-NEXT: s_and_b32 s10, s38, 0xffff
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_mov_b32 s35, s14
+; SI-NEXT: s_mov_b32 s78, s38
; SI-NEXT: s_or_b32 s14, s10, s8
; SI-NEXT: s_mov_b32 s10, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[8:9], 16
-; SI-NEXT: s_and_b32 s8, s82, 0xffff
+; SI-NEXT: s_and_b32 s8, s48, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_mov_b32 s37, s12
+; SI-NEXT: s_mov_b32 s85, s48
; SI-NEXT: s_or_b32 s12, s8, s6
; SI-NEXT: s_mov_b32 s8, s49
; SI-NEXT: s_lshr_b64 s[48:49], s[6:7], 16
-; SI-NEXT: s_and_b32 s6, s84, 0xffff
+; SI-NEXT: s_and_b32 s6, s50, 0xffff
; SI-NEXT: s_mov_b32 s39, s10
+; SI-NEXT: s_mov_b32 s84, s50
; SI-NEXT: s_or_b32 s10, s6, s4
; SI-NEXT: s_mov_b32 s6, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[4:5], 16
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s54, s73, 16
+; SI-NEXT: s_and_b32 s4, s87, 0xffff
; SI-NEXT: s_mov_b32 s49, s8
; SI-NEXT: s_or_b32 s8, s4, s52
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s64, s63, 16
+; SI-NEXT: s_and_b32 s4, s67, 0xffff
; SI-NEXT: s_mov_b32 s51, s6
; SI-NEXT: s_or_b32 s6, s4, s54
-; SI-NEXT: s_and_b32 s4, s98, 0xffff
-; SI-NEXT: s_mov_b32 s16, s63
-; SI-NEXT: s_mov_b32 s63, s61
+; SI-NEXT: s_and_b32 s4, s66, 0xffff
; SI-NEXT: s_mov_b32 s75, s59
; SI-NEXT: s_mov_b32 s73, s57
; SI-NEXT: s_mov_b32 s61, s47
@@ -231839,18 +232122,28 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s4, s4, s64
; SI-NEXT: s_mov_b32 s5, s65
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 16
-; SI-NEXT: s_mov_b32 s53, s66
-; SI-NEXT: s_mov_b32 s55, s67
-; SI-NEXT: s_mov_b32 s65, s17
+; SI-NEXT: s_mov_b32 s53, s86
+; SI-NEXT: s_mov_b32 s83, s67
+; SI-NEXT: s_mov_b32 s55, s96
+; SI-NEXT: s_mov_b32 s96, s98
+; SI-NEXT: s_mov_b32 s86, s66
+; SI-NEXT: s_mov_b32 s65, s97
; SI-NEXT: s_mov_b64 s[66:67], 0
; SI-NEXT: s_branch .LBB111_3
; SI-NEXT: .LBB111_2:
-; SI-NEXT: s_mov_b32 s16, s63
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s86, s66
+; SI-NEXT: s_mov_b32 s83, s67
+; SI-NEXT: s_mov_b32 s84, s50
+; SI-NEXT: s_mov_b32 s85, s48
+; SI-NEXT: s_mov_b32 s78, s38
+; SI-NEXT: s_mov_b32 s76, s36
; SI-NEXT: s_mov_b64 s[66:67], -1
+; SI-NEXT: v_writelane_b32 v33, s4, 20
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: v_writelane_b32 v33, s5, 21
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr96
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr60
@@ -231881,22 +232174,25 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: .LBB111_3: ; %Flow
; SI-NEXT: s_and_b64 s[66:67], s[66:67], exec
-; SI-NEXT: s_cselect_b32 s17, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s17, 1
+; SI-NEXT: s_cselect_b32 s66, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s66, 1
+; SI-NEXT: v_readlane_b32 s66, v33, 20
+; SI-NEXT: v_readlane_b32 s67, v33, 21
; SI-NEXT: s_cbranch_scc1 .LBB111_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s98, s98, 3
-; SI-NEXT: s_and_b32 s4, s98, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s16, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v33, 16
+; SI-NEXT: s_add_i32 s4, s86, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_readlane_b32 s5, v33, 17
; SI-NEXT: s_add_i32 s5, s5, 3
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_lshl_b32 s6, s65, 16
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: s_add_i32 s96, s96, 3
-; SI-NEXT: v_readlane_b32 s6, v33, 15
-; SI-NEXT: s_and_b32 s7, s96, 0xffff
+; SI-NEXT: s_add_i32 s6, s83, 3
+; SI-NEXT: s_and_b32 s7, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v33, 16
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: v_readlane_b32 s7, v33, 14
@@ -231904,9 +232200,9 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_lshl_b32 s8, s55, 16
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_add_i32 s86, s86, 3
+; SI-NEXT: s_add_i32 s8, s87, 3
+; SI-NEXT: s_and_b32 s9, s8, 0xffff
; SI-NEXT: v_readlane_b32 s8, v33, 13
-; SI-NEXT: s_and_b32 s9, s86, 0xffff
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_or_b32 s8, s8, s9
; SI-NEXT: v_readlane_b32 s9, v33, 12
@@ -231914,9 +232210,9 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s10, s53, 16
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_add_i32 s84, s84, 3
+; SI-NEXT: s_add_i32 s10, s84, 3
+; SI-NEXT: s_and_b32 s11, s10, 0xffff
; SI-NEXT: v_readlane_b32 s10, v33, 5
-; SI-NEXT: s_and_b32 s11, s84, 0xffff
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_or_b32 s10, s10, s11
; SI-NEXT: v_readlane_b32 s11, v33, 11
@@ -231924,9 +232220,9 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s12, s51, 16
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_add_i32 s82, s82, 3
+; SI-NEXT: s_add_i32 s12, s85, 3
+; SI-NEXT: s_and_b32 s13, s12, 0xffff
; SI-NEXT: v_readlane_b32 s12, v33, 4
-; SI-NEXT: s_and_b32 s13, s82, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_or_b32 s12, s12, s13
; SI-NEXT: v_readlane_b32 s13, v33, 10
@@ -231934,123 +232230,116 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s14, s49, 16
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: s_add_i32 s14, s78, 3
+; SI-NEXT: s_and_b32 s15, s14, 0xffff
; SI-NEXT: v_readlane_b32 s14, v33, 3
-; SI-NEXT: s_and_b32 s15, s80, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_or_b32 s14, s14, s15
; SI-NEXT: v_readlane_b32 s15, v33, 9
; SI-NEXT: s_add_i32 s15, s15, 3
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s39, 16
-; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: s_add_i32 s71, s71, 3
-; SI-NEXT: v_readlane_b32 s16, v33, 2
-; SI-NEXT: s_and_b32 s17, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_add_i32 s40, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 8
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s37, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s41, s16, 0x30000
-; SI-NEXT: s_add_i32 s69, s69, 3
-; SI-NEXT: v_readlane_b32 s16, v33, 1
-; SI-NEXT: s_and_b32 s17, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_add_i32 s42, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 7
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s35, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s43, s16, 0x30000
-; SI-NEXT: s_add_i32 s68, s68, 3
-; SI-NEXT: v_readlane_b32 s16, v33, 0
-; SI-NEXT: s_and_b32 s17, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_add_i32 s44, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 6
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s31, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_lshl_b32 s40, s39, 16
+; SI-NEXT: s_or_b32 s15, s40, s15
+; SI-NEXT: s_add_i32 s40, s76, 3
+; SI-NEXT: s_and_b32 s41, s40, 0xffff
+; SI-NEXT: v_readlane_b32 s40, v33, 2
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: v_readlane_b32 s41, v33, 8
+; SI-NEXT: s_add_i32 s41, s41, 3
+; SI-NEXT: s_and_b32 s41, s41, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s37, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: v_readlane_b32 s42, v33, 18
+; SI-NEXT: s_add_i32 s42, s42, 3
+; SI-NEXT: s_and_b32 s43, s42, 0xffff
+; SI-NEXT: v_readlane_b32 s42, v33, 1
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: v_readlane_b32 s43, v33, 7
+; SI-NEXT: s_add_i32 s43, s43, 3
+; SI-NEXT: s_and_b32 s43, s43, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s35, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: v_readlane_b32 s44, v33, 15
+; SI-NEXT: s_add_i32 s44, s44, 3
+; SI-NEXT: s_and_b32 s45, s44, 0xffff
+; SI-NEXT: v_readlane_b32 s44, v33, 0
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: v_readlane_b32 s45, v33, 6
+; SI-NEXT: s_add_i32 s45, s45, 3
+; SI-NEXT: s_and_b32 s45, s45, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s31, 16
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s45, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s99, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s28, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s82, 16
+; SI-NEXT: s_or_b32 s28, s46, s28
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_add_i32 s46, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s17, vcc_hi, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s46, s28, 0x30000
+; SI-NEXT: s_and_b32 s28, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s29, vcc_hi, 16
+; SI-NEXT: s_or_b32 s28, s29, s28
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_add_i32 s47, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s97, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s47, s28, 0x30000
+; SI-NEXT: s_and_b32 s26, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s28, s81, 16
+; SI-NEXT: s_or_b32 s26, s28, s26
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_add_i32 s56, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s95, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s56, s26, 0x30000
+; SI-NEXT: s_and_b32 s26, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s27, s95, 16
+; SI-NEXT: s_or_b32 s26, s27, s26
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_add_i32 s57, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s87, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s57, s26, 0x30000
+; SI-NEXT: s_and_b32 s24, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s26, s80, 16
+; SI-NEXT: s_or_b32 s24, s26, s24
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_add_i32 s58, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s93, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s58, s24, 0x30000
+; SI-NEXT: s_and_b32 s24, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s25, s93, 16
+; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_add_i32 s59, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s85, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s59, s24, 0x30000
+; SI-NEXT: s_and_b32 s22, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s71, 16
+; SI-NEXT: s_or_b32 s22, s24, s22
; SI-NEXT: s_add_i32 s23, s23, 3
-; SI-NEXT: s_add_i32 s60, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s91, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s60, s22, 0x30000
+; SI-NEXT: s_and_b32 s22, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s23, s91, 16
+; SI-NEXT: s_or_b32 s22, s23, s22
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s61, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s83, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s61, s22, 0x30000
+; SI-NEXT: s_and_b32 s20, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s22, s70, 16
+; SI-NEXT: s_or_b32 s20, s22, s20
; SI-NEXT: s_add_i32 s21, s21, 3
-; SI-NEXT: s_add_i32 s72, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s89, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s72, s20, 0x30000
+; SI-NEXT: s_and_b32 s20, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s21, s89, 16
+; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_add_i32 s73, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s81, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s73, s20, 0x30000
+; SI-NEXT: s_and_b32 s18, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s20, s69, 16
+; SI-NEXT: s_or_b32 s18, s20, s18
; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_add_i32 s74, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s79, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s75, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 18
+; SI-NEXT: s_add_i32 s74, s18, 0x30000
+; SI-NEXT: s_and_b32 s18, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s19, s79, 16
+; SI-NEXT: s_or_b32 s18, s19, s18
; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s75, s18, 0x30000
; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s70, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_lshl_b32 s18, s68, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s62, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 17
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
+; SI-NEXT: s_and_b32 s16, s17, 0xffff
; SI-NEXT: s_lshl_b32 s17, s77, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
@@ -232063,9 +232352,14 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s13, s13, 0x30000
; SI-NEXT: s_add_i32 s14, s14, 0x30000
; SI-NEXT: s_add_i32 s15, s15, 0x30000
+; SI-NEXT: s_add_i32 s40, s40, 0x30000
+; SI-NEXT: s_add_i32 s41, s41, 0x30000
+; SI-NEXT: s_add_i32 s42, s42, 0x30000
+; SI-NEXT: s_add_i32 s43, s43, 0x30000
+; SI-NEXT: s_add_i32 s44, s44, 0x30000
+; SI-NEXT: s_add_i32 s45, s45, 0x30000
+; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s63, s16, 0x30000
-; SI-NEXT: s_lshr_b64 s[76:77], s[62:63], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[74:75], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[72:73], 16
; SI-NEXT: s_lshr_b64 s[90:91], s[60:61], 16
; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
@@ -232080,6 +232374,8 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 s[52:53], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[54:55], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[66:67], s[62:63], 16
+; SI-NEXT: s_lshr_b64 s[96:97], s[74:75], 16
; SI-NEXT: s_lshr_b32 s77, s63, 16
; SI-NEXT: s_lshr_b32 s79, s75, 16
; SI-NEXT: s_lshr_b32 s89, s73, 16
@@ -232098,13 +232394,13 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s65, s5, 16
; SI-NEXT: .LBB111_5: ; %end
; SI-NEXT: s_and_b32 s16, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s76, 16
+; SI-NEXT: s_lshl_b32 s17, s66, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s63, 0xffff
; SI-NEXT: s_lshl_b32 s18, s77, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s78, 16
+; SI-NEXT: s_lshl_b32 s19, s96, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s75, 0xffff
; SI-NEXT: s_lshl_b32 s20, s79, 16
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
index 59725b5b5dea6c..9d0bd053805b0d 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
@@ -60662,18 +60662,17 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v17, s34, 0
; SI-NEXT: v_writelane_b32 v17, s35, 1
-; SI-NEXT: v_writelane_b32 v17, s36, 2
-; SI-NEXT: v_writelane_b32 v17, s37, 3
-; SI-NEXT: v_writelane_b32 v17, s30, 4
-; SI-NEXT: v_writelane_b32 v17, s31, 5
-; SI-NEXT: v_readfirstlane_b32 s36, v1
-; SI-NEXT: v_readfirstlane_b32 s34, v0
-; SI-NEXT: s_lshr_b32 s30, s29, 16
-; SI-NEXT: s_lshr_b32 s4, s28, 16
+; SI-NEXT: v_writelane_b32 v17, s30, 2
+; SI-NEXT: v_writelane_b32 v17, s31, 3
+; SI-NEXT: v_readfirstlane_b32 s10, v1
+; SI-NEXT: v_readfirstlane_b32 s8, v0
+; SI-NEXT: s_lshr_b32 s4, s29, 16
+; SI-NEXT: s_lshr_b32 s5, s28, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s27, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s26, 16
; SI-NEXT: s_lshr_b32 s95, s25, 16
@@ -60684,18 +60683,22 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_lshr_b32 s90, s20, 16
; SI-NEXT: s_lshr_b32 s89, s19, 16
; SI-NEXT: s_lshr_b32 s88, s18, 16
-; SI-NEXT: s_lshr_b32 s79, s17, 16
-; SI-NEXT: s_lshr_b32 s37, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s36, 16
-; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v2
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s6, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s10, 16
+; SI-NEXT: s_lshr_b32 s12, s8, 16
+; SI-NEXT: v_readfirstlane_b32 s9, v2
+; SI-NEXT: ; implicit-def: $vgpr18 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v18, s7, 0
+; SI-NEXT: v_writelane_b32 v18, s6, 1
; SI-NEXT: s_cbranch_scc0 .LBB93_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_lshl_b32 s74, s16, 16
-; SI-NEXT: s_lshl_b32 s78, s37, 16
+; SI-NEXT: s_lshl_b32 s78, s7, 16
; SI-NEXT: s_lshl_b32 s72, s17, 16
-; SI-NEXT: s_lshl_b32 s77, s79, 16
+; SI-NEXT: s_lshl_b32 s77, s6, 16
; SI-NEXT: s_lshl_b32 s62, s18, 16
; SI-NEXT: s_lshl_b32 s76, s88, 16
; SI-NEXT: s_lshl_b32 s60, s19, 16
@@ -60710,25 +60713,35 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_lshl_b32 s59, s93, 16
; SI-NEXT: s_lshl_b32 s42, s24, 16
; SI-NEXT: s_lshl_b32 s57, s94, 16
-; SI-NEXT: s_lshl_b32 s43, s25, 16
+; SI-NEXT: s_lshl_b32 s45, s25, 16
; SI-NEXT: s_lshl_b32 s47, s95, 16
-; SI-NEXT: s_lshl_b32 s14, s26, 16
-; SI-NEXT: s_lshl_b32 s45, vcc_lo, 16
-; SI-NEXT: s_lshl_b32 s12, s27, 16
-; SI-NEXT: s_lshl_b32 s40, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s15, s28, 16
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_lshl_b32 s41, s4, 16
-; SI-NEXT: s_lshl_b32 s9, s29, 16
-; SI-NEXT: s_lshl_b32 s13, s30, 16
-; SI-NEXT: s_lshl_b32 s10, s34, 16
-; SI-NEXT: s_lshl_b32 s11, s31, 16
-; SI-NEXT: s_lshl_b32 s7, s36, 16
-; SI-NEXT: s_lshl_b32 s8, s35, 16
+; SI-NEXT: s_lshl_b32 s15, s26, 16
+; SI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s13, s27, 16
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_lshl_b32 s11, s28, 16
+; SI-NEXT: s_mov_b32 s6, s5
+; SI-NEXT: s_lshl_b32 s40, s5, 16
+; SI-NEXT: s_lshl_b32 s79, s29, 16
+; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_lshl_b32 s14, s4, 16
+; SI-NEXT: s_mov_b32 s9, s8
+; SI-NEXT: s_lshl_b32 s34, s8, 16
+; SI-NEXT: s_mov_b32 s8, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_mov_b32 s31, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_mov_b32 s30, s35
+; SI-NEXT: s_lshl_b32 s35, s35, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB93_3
; SI-NEXT: .LBB93_2:
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_mov_b32 s31, s10
+; SI-NEXT: s_mov_b32 s30, s35
+; SI-NEXT: s_mov_b32 s9, s8
+; SI-NEXT: s_mov_b32 s8, s12
+; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s5
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr78
@@ -60748,116 +60761,117 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: ; implicit-def: $sgpr59
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr57
-; SI-NEXT: ; implicit-def: $sgpr43
-; SI-NEXT: ; implicit-def: $sgpr47
-; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr45
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; implicit-def: $sgpr47
; SI-NEXT: ; implicit-def: $sgpr15
-; SI-NEXT: ; implicit-def: $sgpr41
-; SI-NEXT: ; implicit-def: $sgpr9
+; SI-NEXT: ; implicit-def: $sgpr43
; SI-NEXT: ; implicit-def: $sgpr13
-; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr41
; SI-NEXT: ; implicit-def: $sgpr11
-; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; implicit-def: $sgpr79
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr35
; SI-NEXT: .LBB93_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s4, s7
-; SI-NEXT: s_mov_b32 s5, s10
-; SI-NEXT: s_mov_b32 s7, s9
-; SI-NEXT: s_mov_b32 s9, s15
-; SI-NEXT: s_mov_b32 s10, s12
-; SI-NEXT: s_mov_b32 s12, s14
-; SI-NEXT: s_mov_b32 s14, s43
-; SI-NEXT: s_mov_b32 s15, s41
+; SI-NEXT: s_mov_b32 s4, s10
+; SI-NEXT: s_mov_b32 s5, s34
+; SI-NEXT: s_mov_b32 s10, s11
+; SI-NEXT: s_mov_b32 s11, s13
+; SI-NEXT: s_mov_b32 s13, s15
+; SI-NEXT: s_mov_b32 s15, s45
+; SI-NEXT: s_mov_b32 s45, s47
; SI-NEXT: s_cbranch_scc1 .LBB93_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s36, s36, 3
-; SI-NEXT: s_and_b32 s4, s36, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_add_i32 s34, s34, 3
+; SI-NEXT: s_add_i32 s4, s31, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s30, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s47, s4, 0x30000
+; SI-NEXT: s_add_i32 s4, s9, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s5, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: s_and_b32 s7, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s30, 16
+; SI-NEXT: s_add_i32 s5, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s4, s7, s4
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s8, s28, 0xffff
+; SI-NEXT: s_add_i32 s7, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s28, 0xffff
; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_or_b32 s6, s6, s8
-; SI-NEXT: s_and_b32 s8, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s9, vcc_hi, 16
+; SI-NEXT: s_add_i32 s6, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s8, vcc_hi, 16
+; SI-NEXT: s_or_b32 s4, s8, s4
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: s_and_b32 s9, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_add_i32 s8, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s9, vcc_lo, 16
+; SI-NEXT: s_or_b32 s4, s9, s4
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s10, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s95, 16
+; SI-NEXT: s_add_i32 s9, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s4, s10, s4
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: s_and_b32 s11, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s12, s94, 16
+; SI-NEXT: s_add_i32 s10, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s11, s94, 16
+; SI-NEXT: s_or_b32 s4, s11, s4
; SI-NEXT: s_add_i32 s23, s23, 3
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_and_b32 s12, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s93, 16
+; SI-NEXT: s_add_i32 s11, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s93, 16
+; SI-NEXT: s_or_b32 s4, s12, s4
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s13, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s14, s92, 16
+; SI-NEXT: s_add_i32 s12, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s92, 16
+; SI-NEXT: s_or_b32 s4, s13, s4
; SI-NEXT: s_add_i32 s21, s21, 3
-; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: s_and_b32 s14, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s91, 16
+; SI-NEXT: s_add_i32 s13, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s14, s91, 16
+; SI-NEXT: s_or_b32 s4, s14, s4
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: s_and_b32 s15, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s90, 16
+; SI-NEXT: s_add_i32 s14, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s15, s90, 16
+; SI-NEXT: s_or_b32 s4, s15, s4
; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_or_b32 s15, s20, s15
-; SI-NEXT: s_and_b32 s19, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s89, 16
+; SI-NEXT: s_add_i32 s15, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s19, s89, 16
+; SI-NEXT: s_or_b32 s4, s19, s4
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s18, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s88, 16
+; SI-NEXT: s_add_i32 s19, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s18, s88, 16
+; SI-NEXT: s_or_b32 s4, s18, s4
+; SI-NEXT: s_add_i32 s18, s4, 0x30000
; SI-NEXT: s_add_i32 s17, s17, 3
-; SI-NEXT: s_or_b32 s18, s20, s18
+; SI-NEXT: v_readlane_b32 s4, v18, 1
; SI-NEXT: s_and_b32 s17, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s79, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s17
+; SI-NEXT: s_add_i32 s17, s4, 0x30000
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_or_b32 s17, s20, s17
+; SI-NEXT: v_readlane_b32 s4, v18, 0
; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s37, 16
-; SI-NEXT: s_or_b32 s16, s20, s16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
-; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_add_i32 s7, s7, 0x30000
-; SI-NEXT: s_add_i32 s6, s6, 0x30000
-; SI-NEXT: s_add_i32 s8, s8, 0x30000
-; SI-NEXT: s_add_i32 s9, s9, 0x30000
-; SI-NEXT: s_add_i32 s10, s10, 0x30000
-; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_add_i32 s12, s12, 0x30000
-; SI-NEXT: s_add_i32 s13, s13, 0x30000
-; SI-NEXT: s_add_i32 s14, s14, 0x30000
-; SI-NEXT: s_add_i32 s15, s15, 0x30000
-; SI-NEXT: s_add_i32 s19, s19, 0x30000
-; SI-NEXT: s_add_i32 s18, s18, 0x30000
-; SI-NEXT: s_add_i32 s17, s17, 0x30000
-; SI-NEXT: s_add_i32 s16, s16, 0x30000
-; SI-NEXT: s_and_b32 s78, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s74, s16, 16
+; SI-NEXT: s_and_b32 s78, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s4, 16
; SI-NEXT: s_and_b32 s77, s17, 0xffff0000
; SI-NEXT: s_lshl_b32 s72, s17, 16
; SI-NEXT: s_and_b32 s76, s18, 0xffff0000
@@ -60874,20 +60888,20 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_lshl_b32 s44, s12, 16
; SI-NEXT: s_and_b32 s57, s11, 0xffff0000
; SI-NEXT: s_lshl_b32 s42, s11, 16
-; SI-NEXT: s_and_b32 s47, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 s14, s10, 16
-; SI-NEXT: s_and_b32 s45, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s12, s9, 16
-; SI-NEXT: s_and_b32 s40, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s10, s8, 16
-; SI-NEXT: s_and_b32 s15, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s6, 16
-; SI-NEXT: s_and_b32 s13, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s11, s5, 0xffff0000
+; SI-NEXT: s_and_b32 s45, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s10, 16
+; SI-NEXT: s_and_b32 s43, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s9, 16
+; SI-NEXT: s_and_b32 s41, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s11, s8, 16
+; SI-NEXT: s_and_b32 s40, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s6, 16
+; SI-NEXT: s_and_b32 s14, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s7, 16
+; SI-NEXT: s_and_b32 s12, s5, 0xffff0000
; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_and_b32 s8, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_and_b32 s35, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s4, s47, 16
; SI-NEXT: .LBB93_5: ; %end
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
@@ -60925,42 +60939,41 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
; SI-NEXT: v_mul_f32_e64 v8, 1.0, s42
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s45
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s15
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s13
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s11
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s40
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s10
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s14
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s79
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s12
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
; SI-NEXT: v_mul_f32_e64 v14, 1.0, s5
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
; SI-NEXT: v_mul_f32_e64 v15, 1.0, s4
-; SI-NEXT: v_readlane_b32 s30, v17, 4
+; SI-NEXT: v_readlane_b32 s30, v17, 2
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_readlane_b32 s31, v17, 5
-; SI-NEXT: v_readlane_b32 s37, v17, 3
-; SI-NEXT: v_readlane_b32 s36, v17, 2
+; SI-NEXT: v_readlane_b32 s31, v17, 3
; SI-NEXT: v_readlane_b32 s35, v17, 1
; SI-NEXT: v_readlane_b32 s34, v17, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -70628,13 +70641,8 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v43, s84, 26
; SI-NEXT: v_writelane_b32 v43, s85, 27
; SI-NEXT: v_writelane_b32 v43, s86, 28
-; SI-NEXT: v_writelane_b32 v43, s87, 29
-; SI-NEXT: v_writelane_b32 v43, s96, 30
-; SI-NEXT: v_writelane_b32 v43, s97, 31
-; SI-NEXT: v_writelane_b32 v43, s98, 32
-; SI-NEXT: v_writelane_b32 v43, s99, 33
-; SI-NEXT: v_writelane_b32 v43, s30, 34
-; SI-NEXT: v_writelane_b32 v43, s31, 35
+; SI-NEXT: v_writelane_b32 v43, s30, 29
+; SI-NEXT: v_writelane_b32 v43, s31, 30
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
@@ -70678,8 +70686,9 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v44, s10, 7
; SI-NEXT: v_readfirstlane_b32 s46, v16
; SI-NEXT: v_writelane_b32 v44, s11, 8
-; SI-NEXT: v_readfirstlane_b32 s60, v2
+; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: v_writelane_b32 v44, s46, 9
+; SI-NEXT: v_writelane_b32 v44, s45, 10
; SI-NEXT: v_readfirstlane_b32 s63, v30
; SI-NEXT: v_readfirstlane_b32 s52, v29
; SI-NEXT: v_readfirstlane_b32 s51, v28
@@ -70699,8 +70708,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s59, v12
; SI-NEXT: v_readfirstlane_b32 s35, v11
; SI-NEXT: v_readfirstlane_b32 s36, v10
-; SI-NEXT: v_readfirstlane_b32 s45, v3
-; SI-NEXT: v_writelane_b32 v44, s60, 10
+; SI-NEXT: v_readfirstlane_b32 s68, v3
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
@@ -70709,73 +70717,70 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s40, v34
; SI-NEXT: v_readfirstlane_b32 s41, v35
; SI-NEXT: v_readfirstlane_b32 s13, v36
-; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_readfirstlane_b32 s5, v38
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s76, v39
-; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_readfirstlane_b32 s31, v48
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s73, v49
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s77, v39
+; SI-NEXT: v_writelane_b32 v44, s77, 11
+; SI-NEXT: v_writelane_b32 v44, s5, 12
+; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s44, v51
-; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: v_readfirstlane_b32 s31, v48
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s43, v53
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_readfirstlane_b32 s73, v49
+; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_readfirstlane_b32 s76, v41
+; SI-NEXT: v_writelane_b32 v44, s76, 13
+; SI-NEXT: v_writelane_b32 v44, s44, 14
; SI-NEXT: v_readfirstlane_b32 s65, v54
-; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_readfirstlane_b32 s66, v55
-; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s55, v40
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s82, v41
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s64, v42
-; SI-NEXT: v_writelane_b32 v44, s76, 11
+; SI-NEXT: v_writelane_b32 v44, s43, 15
+; SI-NEXT: v_writelane_b32 v44, s42, 16
; SI-NEXT: s_cbranch_scc0 .LBB99_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_lshl_b32 s6, s17, 8
-; SI-NEXT: s_or_b32 vcc_hi, s4, s6
+; SI-NEXT: s_or_b32 s88, s4, s6
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s19, 24
; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_and_b32 s6, s24, 0xff
-; SI-NEXT: s_lshl_b32 s62, s25, 8
-; SI-NEXT: s_or_b32 s67, s6, s62
+; SI-NEXT: s_lshl_b32 s60, s25, 8
+; SI-NEXT: s_or_b32 s89, s6, s60
; SI-NEXT: s_and_b32 s6, s26, 0xff
; SI-NEXT: s_and_b32 s72, s72, 0xff
; SI-NEXT: s_lshl_b32 s58, s58, 8
; SI-NEXT: s_and_b32 s56, s56, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s62, s27, 24
+; SI-NEXT: s_lshl_b32 s60, s27, 24
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_or_b32 s58, s72, s58
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s9, s9, 24
-; SI-NEXT: s_or_b32 s6, s62, s6
-; SI-NEXT: s_and_b32 s60, s60, 0xff
-; SI-NEXT: s_lshl_b32 s62, s45, 8
+; SI-NEXT: s_or_b32 s6, s60, s6
+; SI-NEXT: s_and_b32 s45, s45, 0xff
+; SI-NEXT: s_lshl_b32 s60, s68, 8
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_and_b32 s58, s58, 0xffff
-; SI-NEXT: s_or_b32 s87, s9, s56
-; SI-NEXT: s_or_b32 s70, s60, s62
+; SI-NEXT: s_or_b32 s81, s9, s56
+; SI-NEXT: s_or_b32 s45, s45, s60
; SI-NEXT: s_or_b32 s8, s8, s10
; SI-NEXT: s_and_b32 s10, s36, 0xff
; SI-NEXT: s_lshl_b32 s60, s35, 8
-; SI-NEXT: s_or_b32 s9, s58, s87
+; SI-NEXT: s_or_b32 s9, s58, s81
; SI-NEXT: s_and_b32 s56, s57, 0xff
; SI-NEXT: s_lshl_b32 s58, s47, 8
; SI-NEXT: s_and_b32 s46, s46, 0xff
-; SI-NEXT: s_or_b32 s71, s10, s60
+; SI-NEXT: s_or_b32 s90, s10, s60
; SI-NEXT: s_and_b32 s10, s59, 0xff
; SI-NEXT: s_or_b32 s56, s56, s58
; SI-NEXT: s_lshl_b32 s46, s46, 16
@@ -70783,14 +70788,14 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s60, s30, 24
; SI-NEXT: s_and_b32 s56, s56, 0xffff
-; SI-NEXT: s_or_b32 s96, s11, s46
+; SI-NEXT: s_or_b32 s82, s11, s46
; SI-NEXT: s_or_b32 s10, s60, s10
; SI-NEXT: s_and_b32 s60, s50, 0xff
; SI-NEXT: s_lshl_b32 s62, s49, 8
-; SI-NEXT: s_or_b32 s11, s56, s96
+; SI-NEXT: s_or_b32 s11, s56, s82
; SI-NEXT: s_and_b32 s46, s38, 0xff
; SI-NEXT: s_lshl_b32 s56, s37, 8
-; SI-NEXT: s_or_b32 s80, s60, s62
+; SI-NEXT: s_or_b32 vcc_hi, s60, s62
; SI-NEXT: s_and_b32 s60, s39, 0xff
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s56, s61, 0xff
@@ -70800,102 +70805,95 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s58, s34, 24
; SI-NEXT: s_or_b32 s60, s62, s60
; SI-NEXT: s_and_b32 s62, s54, 0xff
-; SI-NEXT: s_lshl_b32 s77, s53, 8
+; SI-NEXT: s_lshl_b32 s78, s53, 8
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s97, s58, s56
-; SI-NEXT: s_or_b32 s81, s62, s77
+; SI-NEXT: s_or_b32 s83, s58, s56
+; SI-NEXT: s_or_b32 s67, s62, s78
; SI-NEXT: s_and_b32 s62, s51, 0xff
-; SI-NEXT: s_and_b32 s76, s76, 0xff
-; SI-NEXT: s_mov_b32 s79, s5
+; SI-NEXT: s_and_b32 s77, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_mov_b32 s72, s61
-; SI-NEXT: s_or_b32 s61, s46, s97
+; SI-NEXT: s_or_b32 s61, s46, s83
; SI-NEXT: s_and_b32 s46, s63, 0xff
; SI-NEXT: s_lshl_b32 s56, s64, 8
; SI-NEXT: s_lshl_b32 s62, s62, 16
-; SI-NEXT: s_lshl_b32 s77, s52, 24
-; SI-NEXT: s_or_b32 s84, s76, s5
+; SI-NEXT: s_lshl_b32 s78, s52, 24
+; SI-NEXT: s_or_b32 s70, s77, s5
; SI-NEXT: s_and_b32 s5, s20, 0xff
-; SI-NEXT: s_lshl_b32 s76, s21, 8
+; SI-NEXT: s_lshl_b32 s77, s21, 8
; SI-NEXT: s_or_b32 s46, s46, s56
-; SI-NEXT: s_and_b32 s56, s82, 0xff
-; SI-NEXT: s_or_b32 s62, s77, s62
-; SI-NEXT: s_and_b32 s77, s66, 0xff
-; SI-NEXT: s_lshl_b32 s78, s65, 8
-; SI-NEXT: s_or_b32 s5, s5, s76
-; SI-NEXT: s_and_b32 s76, s22, 0xff
+; SI-NEXT: s_and_b32 s56, s76, 0xff
+; SI-NEXT: s_or_b32 s62, s78, s62
+; SI-NEXT: s_and_b32 s78, s66, 0xff
+; SI-NEXT: s_lshl_b32 s79, s65, 8
+; SI-NEXT: s_or_b32 s5, s5, s77
+; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_and_b32 s74, s74, 0xff
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s58, s55, 24
; SI-NEXT: s_mov_b32 s91, s14
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_or_b32 s83, s77, s78
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_lshl_b32 s77, s23, 24
+; SI-NEXT: s_or_b32 s69, s78, s79
+; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_lshl_b32 s74, s74, 16
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s98, s58, s56
+; SI-NEXT: s_or_b32 s84, s58, s56
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s85, s77, s76
-; SI-NEXT: s_and_b32 s76, s28, 0xff
-; SI-NEXT: s_lshl_b32 s77, s29, 8
-; SI-NEXT: s_or_b32 s86, s7, s74
+; SI-NEXT: s_or_b32 s71, s78, s77
+; SI-NEXT: s_and_b32 s77, s28, 0xff
+; SI-NEXT: s_lshl_b32 s78, s29, 8
+; SI-NEXT: s_or_b32 s80, s7, s74
; SI-NEXT: s_mov_b32 s74, s63
-; SI-NEXT: s_or_b32 s63, s46, s98
-; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_or_b32 s63, s46, s84
; SI-NEXT: s_and_b32 s44, s44, 0xff
; SI-NEXT: s_lshl_b32 s46, s75, 8
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_and_b32 s41, s41, 0xff
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_lshl_b32 s40, s40, 8
-; SI-NEXT: s_or_b32 s78, s12, s14
+; SI-NEXT: s_or_b32 s86, s12, s14
; SI-NEXT: s_and_b32 s12, s15, 0xff
-; SI-NEXT: s_or_b32 s76, s76, s77
+; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_or_b32 s44, s44, s46
; SI-NEXT: s_and_b32 s46, s73, 0xff
-; SI-NEXT: s_mov_b32 s89, s43
; SI-NEXT: s_and_b32 s43, s43, 0xff
; SI-NEXT: s_or_b32 s40, s41, s40
; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_lshl_b32 s13, s13, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_and_b32 s76, s76, 0xffff
+; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_lshl_b32 s46, s46, 16
; SI-NEXT: s_lshl_b32 s56, s31, 24
; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_mov_b32 s90, s42
; SI-NEXT: s_lshl_b32 s42, s42, 24
-; SI-NEXT: s_or_b32 s68, s13, s12
+; SI-NEXT: s_or_b32 s78, s13, s12
; SI-NEXT: s_and_b32 s12, s40, 0xffff
-; SI-NEXT: s_or_b32 s5, s5, s85
-; SI-NEXT: s_or_b32 s7, s76, s86
-; SI-NEXT: s_or_b32 s99, s56, s46
+; SI-NEXT: s_or_b32 s5, s5, s71
+; SI-NEXT: s_or_b32 s7, s77, s80
+; SI-NEXT: s_or_b32 s85, s56, s46
; SI-NEXT: s_or_b32 s76, s42, s43
; SI-NEXT: s_and_b32 s42, s44, 0xffff
-; SI-NEXT: s_or_b32 s69, s12, s78
-; SI-NEXT: s_and_b32 s12, vcc_hi, 0xffff
-; SI-NEXT: s_or_b32 s77, s42, s99
+; SI-NEXT: s_or_b32 s79, s12, s86
+; SI-NEXT: s_and_b32 s12, s88, 0xffff
+; SI-NEXT: s_or_b32 s77, s42, s85
+; SI-NEXT: s_and_b32 s14, s45, 0xffff
; SI-NEXT: s_or_b32 s42, s12, s4
-; SI-NEXT: s_mov_b32 s12, s45
; SI-NEXT: s_lshr_b64 s[44:45], s[4:5], 16
; SI-NEXT: s_mov_b32 s4, s47
; SI-NEXT: s_lshr_b64 s[46:47], s[6:7], 16
-; SI-NEXT: s_and_b32 s13, s67, 0xffff
-; SI-NEXT: s_and_b32 s14, s70, 0xffff
-; SI-NEXT: s_and_b32 s58, s71, 0xffff
-; SI-NEXT: s_and_b32 vcc_hi, s80, 0xffff
-; SI-NEXT: s_and_b32 s67, s81, 0xffff
+; SI-NEXT: s_and_b32 s13, s89, 0xffff
+; SI-NEXT: s_and_b32 s58, s90, 0xffff
+; SI-NEXT: s_and_b32 s88, vcc_hi, 0xffff
+; SI-NEXT: s_and_b32 s89, s67, 0xffff
; SI-NEXT: s_mov_b32 s47, s4
; SI-NEXT: s_mov_b32 s4, s57
; SI-NEXT: s_lshr_b64 s[56:57], s[8:9], 16
; SI-NEXT: s_mov_b32 s95, s15
-; SI-NEXT: s_and_b32 s70, s83, 0xffff
-; SI-NEXT: s_mov_b32 s45, s12
; SI-NEXT: s_or_b32 s40, s13, s6
; SI-NEXT: s_or_b32 s14, s14, s8
; SI-NEXT: s_mov_b32 s15, s9
@@ -70904,62 +70902,59 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_mov_b32 s13, s11
; SI-NEXT: s_mov_b32 s4, s59
; SI-NEXT: s_lshr_b64 s[58:59], s[10:11], 16
-; SI-NEXT: s_or_b32 s10, vcc_hi, s60
+; SI-NEXT: s_or_b32 s10, s88, s60
; SI-NEXT: s_mov_b32 s11, s61
; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_or_b32 s8, s67, s62
+; SI-NEXT: s_or_b32 s8, s89, s62
; SI-NEXT: s_mov_b32 s9, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
-; SI-NEXT: s_and_b32 s71, s84, 0xffff
+; SI-NEXT: s_and_b32 s90, s69, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s70, 0xffff
; SI-NEXT: s_mov_b32 s59, s4
; SI-NEXT: s_mov_b32 s61, s72
; SI-NEXT: s_mov_b32 s63, s74
-; SI-NEXT: s_or_b32 s6, s70, s76
; SI-NEXT: s_mov_b32 s4, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[76:77], 16
-; SI-NEXT: s_mov_b32 s76, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[68:69], 16
+; SI-NEXT: s_mov_b32 s45, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[78:79], 16
; SI-NEXT: s_mov_b32 s43, s5
; SI-NEXT: s_mov_b32 s41, s7
+; SI-NEXT: s_or_b32 s6, s90, s76
; SI-NEXT: s_mov_b32 s7, s77
; SI-NEXT: s_mov_b32 s73, s4
-; SI-NEXT: s_or_b32 s4, s71, s68
-; SI-NEXT: s_mov_b32 s5, s69
-; SI-NEXT: s_mov_b32 s75, s76
-; SI-NEXT: s_lshr_b32 s71, s85, 16
-; SI-NEXT: s_lshr_b32 s70, s86, 16
-; SI-NEXT: s_lshr_b32 s68, s87, 16
-; SI-NEXT: s_lshr_b32 s69, s96, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s97, 16
-; SI-NEXT: s_lshr_b32 s67, s98, 16
-; SI-NEXT: s_lshr_b32 s80, s99, 16
-; SI-NEXT: s_lshr_b32 s81, s78, 16
+; SI-NEXT: s_or_b32 s4, vcc_hi, s78
+; SI-NEXT: s_mov_b32 s5, s79
+; SI-NEXT: s_mov_b32 s75, s45
+; SI-NEXT: s_lshr_b32 s89, s71, 16
+; SI-NEXT: s_lshr_b32 s88, s80, 16
+; SI-NEXT: s_lshr_b32 s79, s81, 16
+; SI-NEXT: s_lshr_b32 s78, s82, 16
+; SI-NEXT: s_lshr_b32 vcc_hi, s83, 16
+; SI-NEXT: s_lshr_b32 s67, s84, 16
+; SI-NEXT: s_lshr_b32 s90, s85, 16
+; SI-NEXT: s_lshr_b32 s45, s86, 16
; SI-NEXT: s_mov_b64 s[76:77], 0
; SI-NEXT: s_branch .LBB99_3
; SI-NEXT: .LBB99_2:
-; SI-NEXT: s_mov_b32 s79, s5
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_mov_b32 s95, s15
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_mov_b32 s91, s14
-; SI-NEXT: s_mov_b32 s90, s42
-; SI-NEXT: s_mov_b32 s89, s43
-; SI-NEXT: s_mov_b32 s88, s44
; SI-NEXT: s_mov_b64 s[76:77], -1
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr71
+; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr68
+; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $vcc_hi
@@ -70968,10 +70963,10 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr45
; SI-NEXT: .LBB99_3: ; %Flow
; SI-NEXT: s_and_b64 s[76:77], s[76:77], exec
; SI-NEXT: s_cselect_b32 s76, 1, 0
@@ -70980,10 +70975,11 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: v_readlane_b32 s4, v44, 11
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s5, s79, 8
+; SI-NEXT: s_and_b32 s5, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v44, 12
+; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_add_i32 s6, s95, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_lshl_b32 s5, vcc_lo, 24
@@ -71006,16 +71002,19 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s6, s66, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s65, 8
-; SI-NEXT: s_add_i32 s8, s89, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s90, 24
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s8
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s7, s88, 3
+; SI-NEXT: s_and_b32 s7, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 16
+; SI-NEXT: s_lshl_b32 s8, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 15
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: v_readlane_b32 s7, v44, 14
+; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s75, 8
; SI-NEXT: s_add_i32 s9, s73, 3
@@ -71042,15 +71041,16 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s9, s63, 3
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s64, 8
-; SI-NEXT: s_add_i32 s82, s82, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s11, s82, 0xff
; SI-NEXT: s_addk_i32 s9, 0x300
-; SI-NEXT: s_lshl_b32 s10, s55, 24
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_or_b32 s10, s10, s11
-; SI-NEXT: s_or_b32 s9, s10, s9
+; SI-NEXT: s_and_b32 s10, s9, 0xffff
+; SI-NEXT: v_readlane_b32 s9, v44, 13
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: s_lshl_b32 s11, s55, 24
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s9, s11, s9
+; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_add_i32 s10, s50, 3
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s11, s49, 8
@@ -71104,7 +71104,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readlane_b32 s14, v44, 10
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s15, s45, 8
+; SI-NEXT: s_lshl_b32 s15, s68, 8
; SI-NEXT: s_or_b32 s14, s15, s14
; SI-NEXT: s_addk_i32 s14, 0x300
; SI-NEXT: s_and_b32 s15, s14, 0xffff
@@ -71141,49 +71141,50 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_and_b32 s24, s24, 0xffff
; SI-NEXT: s_or_b32 s25, s25, s26
+; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_or_b32 s15, s41, s15
; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s15, s15, s40
; SI-NEXT: s_add_i32 s40, s24, 0x3000000
; SI-NEXT: s_and_b32 s24, s28, 0xff
; SI-NEXT: s_lshl_b32 s25, s29, 8
-; SI-NEXT: s_and_b32 s16, s16, 0xff
-; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s24, 0x300
+; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s19, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s25, s24, 0xffff
-; SI-NEXT: v_readlane_b32 s24, v44, 0
+; SI-NEXT: s_addk_i32 s24, 0x300
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_lshl_b32 s26, s24, 24
-; SI-NEXT: v_readlane_b32 s24, v44, 1
+; SI-NEXT: s_and_b32 s25, s24, 0xffff
+; SI-NEXT: v_readlane_b32 s24, v44, 0
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: s_lshl_b32 s26, s24, 24
+; SI-NEXT: v_readlane_b32 s24, v44, 1
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
; SI-NEXT: s_and_b32 s16, s20, 0xff
; SI-NEXT: s_lshl_b32 s17, s21, 8
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_and_b32 s24, s24, 0xff
+; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s22, 0xff
-; SI-NEXT: s_lshl_b32 s24, s24, 16
+; SI-NEXT: s_and_b32 s24, s24, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s23, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_or_b32 s24, s26, s24
+; SI-NEXT: s_lshl_b32 s24, s24, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_or_b32 s24, s26, s24
; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -71197,7 +71198,6 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
; SI-NEXT: s_add_i32 s41, s24, 0x3000000
-; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[46:47], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[56:57], s[14:15], 16
@@ -71206,14 +71206,14 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshr_b64 s[62:63], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[72:73], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b32 s71, s43, 16
-; SI-NEXT: s_lshr_b32 s70, s41, 16
-; SI-NEXT: s_lshr_b32 s68, s15, 16
-; SI-NEXT: s_lshr_b32 s69, s13, 16
+; SI-NEXT: s_lshr_b32 s89, s43, 16
+; SI-NEXT: s_lshr_b32 s88, s41, 16
+; SI-NEXT: s_lshr_b32 s79, s15, 16
+; SI-NEXT: s_lshr_b32 s78, s13, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s11, 16
; SI-NEXT: s_lshr_b32 s67, s9, 16
-; SI-NEXT: s_lshr_b32 s80, s7, 16
-; SI-NEXT: s_lshr_b32 s81, s5, 16
+; SI-NEXT: s_lshr_b32 s90, s7, 16
+; SI-NEXT: s_lshr_b32 s45, s5, 16
; SI-NEXT: .LBB99_5: ; %end
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
@@ -71222,25 +71222,25 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s17, s44, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s71, 16
+; SI-NEXT: s_lshl_b32 s18, s89, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s40, 0xffff
; SI-NEXT: s_lshl_b32 s19, s46, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s41, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s70, 16
+; SI-NEXT: s_lshl_b32 s20, s88, 16
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_lshl_b32 s20, s56, 16
; SI-NEXT: s_or_b32 s14, s14, s20
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s68, 16
+; SI-NEXT: s_lshl_b32 s20, s79, 16
; SI-NEXT: s_or_b32 s15, s15, s20
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_lshl_b32 s20, s58, 16
; SI-NEXT: s_or_b32 s12, s12, s20
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s69, 16
+; SI-NEXT: s_lshl_b32 s20, s78, 16
; SI-NEXT: s_or_b32 s13, s13, s20
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_lshl_b32 s20, s60, 16
@@ -71258,15 +71258,15 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s20, s72, 16
; SI-NEXT: s_or_b32 s6, s6, s20
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s80, 16
+; SI-NEXT: s_lshl_b32 s20, s90, 16
; SI-NEXT: s_or_b32 s7, s7, s20
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s20, s74, 16
; SI-NEXT: s_or_b32 s4, s4, s20
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s81, 16
+; SI-NEXT: s_lshl_b32 s20, s45, 16
; SI-NEXT: s_or_b32 s5, s5, s20
-; SI-NEXT: v_readlane_b32 s30, v43, 34
+; SI-NEXT: v_readlane_b32 s30, v43, 29
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s17
; SI-NEXT: v_mov_b32_e32 v2, s18
@@ -71283,12 +71283,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_mov_b32_e32 v13, s7
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_readlane_b32 s31, v43, 35
-; SI-NEXT: v_readlane_b32 s99, v43, 33
-; SI-NEXT: v_readlane_b32 s98, v43, 32
-; SI-NEXT: v_readlane_b32 s97, v43, 31
-; SI-NEXT: v_readlane_b32 s96, v43, 30
-; SI-NEXT: v_readlane_b32 s87, v43, 29
+; SI-NEXT: v_readlane_b32 s31, v43, 30
; SI-NEXT: v_readlane_b32 s86, v43, 28
; SI-NEXT: v_readlane_b32 s85, v43, 27
; SI-NEXT: v_readlane_b32 s84, v43, 26
@@ -78820,9 +78815,8 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_writelane_b32 v18, s55, 13
; SI-NEXT: v_writelane_b32 v18, s64, 14
; SI-NEXT: v_writelane_b32 v18, s65, 15
-; SI-NEXT: v_writelane_b32 v18, s66, 16
-; SI-NEXT: v_writelane_b32 v18, s30, 17
-; SI-NEXT: v_writelane_b32 v18, s31, 18
+; SI-NEXT: v_writelane_b32 v18, s30, 16
+; SI-NEXT: v_writelane_b32 v18, s31, 17
; SI-NEXT: v_readfirstlane_b32 s31, v2
; SI-NEXT: v_readfirstlane_b32 s44, v1
; SI-NEXT: s_lshr_b32 s75, s29, 16
@@ -78840,7 +78834,7 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_lshr_b32 s45, s17, 16
; SI-NEXT: s_lshr_b32 s47, s16, 16
; SI-NEXT: s_lshr_b32 s49, s31, 16
-; SI-NEXT: s_lshr_b32 s50, s44, 16
+; SI-NEXT: s_lshr_b32 s46, s44, 16
; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: s_cmp_lg_u32 s4, 0
; SI-NEXT: ; implicit-def: $vgpr19 : SGPR spill to VGPR lane
@@ -78898,6 +78892,9 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_and_b32 s4, s23, 0xffff
; SI-NEXT: s_lshl_b32 s5, s38, 16
; SI-NEXT: s_or_b32 s11, s4, s5
+; SI-NEXT: s_lshr_b64 s[4:5], s[10:11], 24
+; SI-NEXT: v_writelane_b32 v19, s4, 18
+; SI-NEXT: v_writelane_b32 v19, s5, 19
; SI-NEXT: s_and_b32 s4, s24, 0xffff
; SI-NEXT: s_lshl_b32 s5, s63, 16
; SI-NEXT: s_or_b32 s12, s4, s5
@@ -78917,17 +78914,18 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_lshl_b32 s5, s75, 16
; SI-NEXT: s_or_b32 s7, s4, s5
; SI-NEXT: s_and_b32 s4, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s50, 16
+; SI-NEXT: s_lshl_b32 s5, s46, 16
; SI-NEXT: s_mov_b32 s76, s44
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s5, s31, 0xffff
; SI-NEXT: s_lshl_b32 s44, s49, 16
; SI-NEXT: s_or_b32 s5, s5, s44
; SI-NEXT: s_lshr_b32 s44, s43, 8
-; SI-NEXT: v_writelane_b32 v19, s44, 19
+; SI-NEXT: v_writelane_b32 v19, s44, 21
; SI-NEXT: s_lshr_b32 s44, s41, 8
-; SI-NEXT: v_writelane_b32 v19, s44, 18
-; SI-NEXT: s_bfe_u32 s51, s45, 0x80008
+; SI-NEXT: s_mov_b32 s78, s46
+; SI-NEXT: v_writelane_b32 v19, s44, 20
+; SI-NEXT: s_bfe_u32 s50, s45, 0x80008
; SI-NEXT: s_mov_b32 s46, s45
; SI-NEXT: s_lshr_b64 s[44:45], s[12:13], 8
; SI-NEXT: s_mov_b32 s45, s46
@@ -78951,11 +78949,10 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_mov_b32 s59, s72
; SI-NEXT: s_mov_b32 s72, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[4:5], 24
-; SI-NEXT: s_bfe_u32 s65, s75, 0x80008
+; SI-NEXT: s_bfe_u32 s64, s75, 0x80008
; SI-NEXT: s_mov_b32 vcc_hi, s72
; SI-NEXT: s_mov_b32 s72, s75
; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[10:11], 24
; SI-NEXT: s_lshr_b64 s[90:91], s[10:11], 16
; SI-NEXT: s_lshr_b64 s[94:95], s[10:11], 8
; SI-NEXT: s_lshr_b64 s[88:89], s[12:13], 24
@@ -78969,47 +78966,52 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_lshr_b32 s89, s9, 8
; SI-NEXT: s_lshr_b32 s79, s7, 8
; SI-NEXT: s_lshr_b32 s77, s5, 8
-; SI-NEXT: s_bfe_u32 s52, s36, 0x80008
-; SI-NEXT: s_bfe_u32 s53, s37, 0x80008
-; SI-NEXT: s_bfe_u32 s54, s38, 0x80008
-; SI-NEXT: s_bfe_u32 s55, s39, 0x80008
-; SI-NEXT: s_bfe_u32 s64, s48, 0x80008
-; SI-NEXT: s_bfe_u32 s66, s49, 0x80008
+; SI-NEXT: s_bfe_u32 s51, s36, 0x80008
+; SI-NEXT: s_bfe_u32 s52, s37, 0x80008
+; SI-NEXT: s_bfe_u32 s53, s38, 0x80008
+; SI-NEXT: s_bfe_u32 s54, s39, 0x80008
+; SI-NEXT: s_bfe_u32 s55, s48, 0x80008
+; SI-NEXT: s_bfe_u32 s65, s49, 0x80008
; SI-NEXT: s_mov_b32 s73, s34
; SI-NEXT: s_mov_b64 s[34:35], 0
; SI-NEXT: s_branch .LBB105_3
; SI-NEXT: .LBB105_2:
; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: s_mov_b32 s76, s44
+; SI-NEXT: s_mov_b32 s78, s46
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v19, s4, 0
; SI-NEXT: v_writelane_b32 v19, s5, 1
; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: s_mov_b64 s[34:35], -1
+; SI-NEXT: s_mov_b32 s76, s44
; SI-NEXT: v_writelane_b32 v19, s4, 2
; SI-NEXT: v_writelane_b32 v19, s5, 3
; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b64 s[34:35], -1
+; SI-NEXT: v_writelane_b32 v19, s4, 4
+; SI-NEXT: v_writelane_b32 v19, s5, 5
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr42
-; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr50
; SI-NEXT: ; implicit-def: $sgpr40
-; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr51
; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $sgpr53
+; SI-NEXT: ; implicit-def: $sgpr52
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr93
-; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr91
-; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr54
; SI-NEXT: ; implicit-def: $sgpr89
-; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr66
+; SI-NEXT: ; implicit-def: $sgpr65
; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr90
-; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr44
; SI-NEXT: ; implicit-def: $sgpr92
@@ -79025,11 +79027,6 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: v_writelane_b32 v19, s4, 4
-; SI-NEXT: v_writelane_b32 v19, s5, 5
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: v_writelane_b32 v19, s4, 6
; SI-NEXT: v_writelane_b32 v19, s5, 7
; SI-NEXT: ; implicit-def: $sgpr4
@@ -79050,6 +79047,9 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_writelane_b32 v19, s4, 16
; SI-NEXT: v_writelane_b32 v19, s5, 17
; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v19, s4, 18
+; SI-NEXT: v_writelane_b32 v19, s5, 19
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB105_3: ; %Flow
; SI-NEXT: s_and_b64 s[34:35], s[34:35], exec
; SI-NEXT: s_cselect_b32 s34, 1, 0
@@ -79057,7 +79057,7 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_cbranch_scc1 .LBB105_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: v_cvt_f32_f16_e32 v2, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s50
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s78
; SI-NEXT: v_cvt_f32_f16_e32 v3, s49
; SI-NEXT: v_cvt_f32_f16_e32 v9, s47
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
@@ -79258,9 +79258,11 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_bfe_u32 v7, v1, 8, 8
; SI-NEXT: s_branch .LBB105_6
; SI-NEXT: .LBB105_5:
-; SI-NEXT: v_mov_b32_e32 v2, s75
; SI-NEXT: v_mov_b32_e32 v6, s37
; SI-NEXT: v_mov_b32_e32 v8, s36
+; SI-NEXT: v_readlane_b32 s36, v19, 12
+; SI-NEXT: v_readlane_b32 s37, v19, 13
+; SI-NEXT: v_mov_b32_e32 v2, s75
; SI-NEXT: s_mov_b32 s16, s74
; SI-NEXT: s_mov_b32 s20, s62
; SI-NEXT: s_mov_b32 s18, s72
@@ -79276,20 +79278,21 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_readlane_b32 s46, v19, 16
; SI-NEXT: v_readlane_b32 s62, v19, 6
; SI-NEXT: v_readlane_b32 s56, v19, 14
-; SI-NEXT: v_readlane_b32 s36, v19, 12
+; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: v_readlane_b32 s36, v19, 18
; SI-NEXT: v_mov_b32_e32 v1, s49
; SI-NEXT: v_mov_b32_e32 v3, s48
; SI-NEXT: v_mov_b32_e32 v4, s39
; SI-NEXT: v_mov_b32_e32 v5, s38
; SI-NEXT: v_mov_b32_e32 v12, s45
-; SI-NEXT: v_mov_b32_e32 v7, s66
-; SI-NEXT: v_mov_b32_e32 v9, s65
-; SI-NEXT: v_mov_b32_e32 v10, s64
-; SI-NEXT: v_mov_b32_e32 v11, s55
-; SI-NEXT: v_mov_b32_e32 v13, s54
-; SI-NEXT: v_mov_b32_e32 v14, s53
-; SI-NEXT: v_mov_b32_e32 v15, s52
-; SI-NEXT: v_mov_b32_e32 v16, s51
+; SI-NEXT: v_mov_b32_e32 v7, s65
+; SI-NEXT: v_mov_b32_e32 v9, s64
+; SI-NEXT: v_mov_b32_e32 v10, s55
+; SI-NEXT: v_mov_b32_e32 v11, s54
+; SI-NEXT: v_mov_b32_e32 v13, s53
+; SI-NEXT: v_mov_b32_e32 v14, s52
+; SI-NEXT: v_mov_b32_e32 v15, s51
+; SI-NEXT: v_mov_b32_e32 v16, s50
; SI-NEXT: v_readlane_b32 s73, v19, 5
; SI-NEXT: v_readlane_b32 s75, v19, 3
; SI-NEXT: v_readlane_b32 s59, v19, 11
@@ -79298,10 +79301,10 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_readlane_b32 s47, v19, 17
; SI-NEXT: v_readlane_b32 s63, v19, 7
; SI-NEXT: v_readlane_b32 s57, v19, 15
-; SI-NEXT: v_readlane_b32 s17, v19, 18
-; SI-NEXT: v_readlane_b32 s19, v19, 19
-; SI-NEXT: v_readlane_b32 s37, v19, 13
-; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: v_readlane_b32 s17, v19, 20
+; SI-NEXT: v_readlane_b32 s19, v19, 21
+; SI-NEXT: v_readlane_b32 s37, v19, 19
+; SI-NEXT: s_mov_b32 s78, s36
; SI-NEXT: .LBB105_6: ; %end
; SI-NEXT: s_and_b32 s21, s42, 0xff
; SI-NEXT: s_lshl_b32 s23, s34, 8
@@ -79483,11 +79486,10 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v1, s4, v1
-; SI-NEXT: v_readlane_b32 s30, v18, 17
+; SI-NEXT: v_readlane_b32 s30, v18, 16
; SI-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen
; SI-NEXT: buffer_store_dword v1, v0, s[0:3], 0 offen offset:60
-; SI-NEXT: v_readlane_b32 s31, v18, 18
-; SI-NEXT: v_readlane_b32 s66, v18, 16
+; SI-NEXT: v_readlane_b32 s31, v18, 17
; SI-NEXT: v_readlane_b32 s65, v18, 15
; SI-NEXT: v_readlane_b32 s64, v18, 14
; SI-NEXT: v_readlane_b32 s55, v18, 13
@@ -83249,13 +83251,8 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v43, s84, 26
; SI-NEXT: v_writelane_b32 v43, s85, 27
; SI-NEXT: v_writelane_b32 v43, s86, 28
-; SI-NEXT: v_writelane_b32 v43, s87, 29
-; SI-NEXT: v_writelane_b32 v43, s96, 30
-; SI-NEXT: v_writelane_b32 v43, s97, 31
-; SI-NEXT: v_writelane_b32 v43, s98, 32
-; SI-NEXT: v_writelane_b32 v43, s99, 33
-; SI-NEXT: v_writelane_b32 v43, s30, 34
-; SI-NEXT: v_writelane_b32 v43, s31, 35
+; SI-NEXT: v_writelane_b32 v43, s30, 29
+; SI-NEXT: v_writelane_b32 v43, s31, 30
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
@@ -83299,8 +83296,9 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v44, s10, 7
; SI-NEXT: v_readfirstlane_b32 s46, v16
; SI-NEXT: v_writelane_b32 v44, s11, 8
-; SI-NEXT: v_readfirstlane_b32 s60, v2
+; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: v_writelane_b32 v44, s46, 9
+; SI-NEXT: v_writelane_b32 v44, s45, 10
; SI-NEXT: v_readfirstlane_b32 s63, v30
; SI-NEXT: v_readfirstlane_b32 s52, v29
; SI-NEXT: v_readfirstlane_b32 s51, v28
@@ -83320,8 +83318,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s59, v12
; SI-NEXT: v_readfirstlane_b32 s35, v11
; SI-NEXT: v_readfirstlane_b32 s36, v10
-; SI-NEXT: v_readfirstlane_b32 s45, v3
-; SI-NEXT: v_writelane_b32 v44, s60, 10
+; SI-NEXT: v_readfirstlane_b32 s68, v3
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
@@ -83330,73 +83327,70 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s40, v34
; SI-NEXT: v_readfirstlane_b32 s41, v35
; SI-NEXT: v_readfirstlane_b32 s13, v36
-; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_readfirstlane_b32 s5, v38
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s76, v39
-; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_readfirstlane_b32 s31, v48
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s73, v49
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s77, v39
+; SI-NEXT: v_writelane_b32 v44, s77, 11
+; SI-NEXT: v_writelane_b32 v44, s5, 12
+; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s44, v51
-; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: v_readfirstlane_b32 s31, v48
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s43, v53
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_readfirstlane_b32 s73, v49
+; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_readfirstlane_b32 s76, v41
+; SI-NEXT: v_writelane_b32 v44, s76, 13
+; SI-NEXT: v_writelane_b32 v44, s44, 14
; SI-NEXT: v_readfirstlane_b32 s65, v54
-; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_readfirstlane_b32 s66, v55
-; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s55, v40
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s82, v41
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s64, v42
-; SI-NEXT: v_writelane_b32 v44, s76, 11
+; SI-NEXT: v_writelane_b32 v44, s43, 15
+; SI-NEXT: v_writelane_b32 v44, s42, 16
; SI-NEXT: s_cbranch_scc0 .LBB107_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_lshl_b32 s6, s17, 8
-; SI-NEXT: s_or_b32 vcc_hi, s4, s6
+; SI-NEXT: s_or_b32 s88, s4, s6
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s19, 24
; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_and_b32 s6, s24, 0xff
-; SI-NEXT: s_lshl_b32 s62, s25, 8
-; SI-NEXT: s_or_b32 s67, s6, s62
+; SI-NEXT: s_lshl_b32 s60, s25, 8
+; SI-NEXT: s_or_b32 s89, s6, s60
; SI-NEXT: s_and_b32 s6, s26, 0xff
; SI-NEXT: s_and_b32 s72, s72, 0xff
; SI-NEXT: s_lshl_b32 s58, s58, 8
; SI-NEXT: s_and_b32 s56, s56, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s62, s27, 24
+; SI-NEXT: s_lshl_b32 s60, s27, 24
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_or_b32 s58, s72, s58
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s9, s9, 24
-; SI-NEXT: s_or_b32 s6, s62, s6
-; SI-NEXT: s_and_b32 s60, s60, 0xff
-; SI-NEXT: s_lshl_b32 s62, s45, 8
+; SI-NEXT: s_or_b32 s6, s60, s6
+; SI-NEXT: s_and_b32 s45, s45, 0xff
+; SI-NEXT: s_lshl_b32 s60, s68, 8
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_and_b32 s58, s58, 0xffff
-; SI-NEXT: s_or_b32 s87, s9, s56
-; SI-NEXT: s_or_b32 s70, s60, s62
+; SI-NEXT: s_or_b32 s81, s9, s56
+; SI-NEXT: s_or_b32 s45, s45, s60
; SI-NEXT: s_or_b32 s8, s8, s10
; SI-NEXT: s_and_b32 s10, s36, 0xff
; SI-NEXT: s_lshl_b32 s60, s35, 8
-; SI-NEXT: s_or_b32 s9, s58, s87
+; SI-NEXT: s_or_b32 s9, s58, s81
; SI-NEXT: s_and_b32 s56, s57, 0xff
; SI-NEXT: s_lshl_b32 s58, s47, 8
; SI-NEXT: s_and_b32 s46, s46, 0xff
-; SI-NEXT: s_or_b32 s71, s10, s60
+; SI-NEXT: s_or_b32 s90, s10, s60
; SI-NEXT: s_and_b32 s10, s59, 0xff
; SI-NEXT: s_or_b32 s56, s56, s58
; SI-NEXT: s_lshl_b32 s46, s46, 16
@@ -83404,14 +83398,14 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s60, s30, 24
; SI-NEXT: s_and_b32 s56, s56, 0xffff
-; SI-NEXT: s_or_b32 s96, s11, s46
+; SI-NEXT: s_or_b32 s82, s11, s46
; SI-NEXT: s_or_b32 s10, s60, s10
; SI-NEXT: s_and_b32 s60, s50, 0xff
; SI-NEXT: s_lshl_b32 s62, s49, 8
-; SI-NEXT: s_or_b32 s11, s56, s96
+; SI-NEXT: s_or_b32 s11, s56, s82
; SI-NEXT: s_and_b32 s46, s38, 0xff
; SI-NEXT: s_lshl_b32 s56, s37, 8
-; SI-NEXT: s_or_b32 s80, s60, s62
+; SI-NEXT: s_or_b32 vcc_hi, s60, s62
; SI-NEXT: s_and_b32 s60, s39, 0xff
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s56, s61, 0xff
@@ -83421,102 +83415,95 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s58, s34, 24
; SI-NEXT: s_or_b32 s60, s62, s60
; SI-NEXT: s_and_b32 s62, s54, 0xff
-; SI-NEXT: s_lshl_b32 s77, s53, 8
+; SI-NEXT: s_lshl_b32 s78, s53, 8
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s97, s58, s56
-; SI-NEXT: s_or_b32 s81, s62, s77
+; SI-NEXT: s_or_b32 s83, s58, s56
+; SI-NEXT: s_or_b32 s67, s62, s78
; SI-NEXT: s_and_b32 s62, s51, 0xff
-; SI-NEXT: s_and_b32 s76, s76, 0xff
-; SI-NEXT: s_mov_b32 s79, s5
+; SI-NEXT: s_and_b32 s77, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_mov_b32 s72, s61
-; SI-NEXT: s_or_b32 s61, s46, s97
+; SI-NEXT: s_or_b32 s61, s46, s83
; SI-NEXT: s_and_b32 s46, s63, 0xff
; SI-NEXT: s_lshl_b32 s56, s64, 8
; SI-NEXT: s_lshl_b32 s62, s62, 16
-; SI-NEXT: s_lshl_b32 s77, s52, 24
-; SI-NEXT: s_or_b32 s84, s76, s5
+; SI-NEXT: s_lshl_b32 s78, s52, 24
+; SI-NEXT: s_or_b32 s70, s77, s5
; SI-NEXT: s_and_b32 s5, s20, 0xff
-; SI-NEXT: s_lshl_b32 s76, s21, 8
+; SI-NEXT: s_lshl_b32 s77, s21, 8
; SI-NEXT: s_or_b32 s46, s46, s56
-; SI-NEXT: s_and_b32 s56, s82, 0xff
-; SI-NEXT: s_or_b32 s62, s77, s62
-; SI-NEXT: s_and_b32 s77, s66, 0xff
-; SI-NEXT: s_lshl_b32 s78, s65, 8
-; SI-NEXT: s_or_b32 s5, s5, s76
-; SI-NEXT: s_and_b32 s76, s22, 0xff
+; SI-NEXT: s_and_b32 s56, s76, 0xff
+; SI-NEXT: s_or_b32 s62, s78, s62
+; SI-NEXT: s_and_b32 s78, s66, 0xff
+; SI-NEXT: s_lshl_b32 s79, s65, 8
+; SI-NEXT: s_or_b32 s5, s5, s77
+; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_and_b32 s74, s74, 0xff
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s58, s55, 24
; SI-NEXT: s_mov_b32 s91, s14
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_or_b32 s83, s77, s78
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_lshl_b32 s77, s23, 24
+; SI-NEXT: s_or_b32 s69, s78, s79
+; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_lshl_b32 s74, s74, 16
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s98, s58, s56
+; SI-NEXT: s_or_b32 s84, s58, s56
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s85, s77, s76
-; SI-NEXT: s_and_b32 s76, s28, 0xff
-; SI-NEXT: s_lshl_b32 s77, s29, 8
-; SI-NEXT: s_or_b32 s86, s7, s74
+; SI-NEXT: s_or_b32 s71, s78, s77
+; SI-NEXT: s_and_b32 s77, s28, 0xff
+; SI-NEXT: s_lshl_b32 s78, s29, 8
+; SI-NEXT: s_or_b32 s80, s7, s74
; SI-NEXT: s_mov_b32 s74, s63
-; SI-NEXT: s_or_b32 s63, s46, s98
-; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_or_b32 s63, s46, s84
; SI-NEXT: s_and_b32 s44, s44, 0xff
; SI-NEXT: s_lshl_b32 s46, s75, 8
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_and_b32 s41, s41, 0xff
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_lshl_b32 s40, s40, 8
-; SI-NEXT: s_or_b32 s78, s12, s14
+; SI-NEXT: s_or_b32 s86, s12, s14
; SI-NEXT: s_and_b32 s12, s15, 0xff
-; SI-NEXT: s_or_b32 s76, s76, s77
+; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_or_b32 s44, s44, s46
; SI-NEXT: s_and_b32 s46, s73, 0xff
-; SI-NEXT: s_mov_b32 s89, s43
; SI-NEXT: s_and_b32 s43, s43, 0xff
; SI-NEXT: s_or_b32 s40, s41, s40
; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_lshl_b32 s13, s13, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_and_b32 s76, s76, 0xffff
+; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_lshl_b32 s46, s46, 16
; SI-NEXT: s_lshl_b32 s56, s31, 24
; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_mov_b32 s90, s42
; SI-NEXT: s_lshl_b32 s42, s42, 24
-; SI-NEXT: s_or_b32 s68, s13, s12
+; SI-NEXT: s_or_b32 s78, s13, s12
; SI-NEXT: s_and_b32 s12, s40, 0xffff
-; SI-NEXT: s_or_b32 s5, s5, s85
-; SI-NEXT: s_or_b32 s7, s76, s86
-; SI-NEXT: s_or_b32 s99, s56, s46
+; SI-NEXT: s_or_b32 s5, s5, s71
+; SI-NEXT: s_or_b32 s7, s77, s80
+; SI-NEXT: s_or_b32 s85, s56, s46
; SI-NEXT: s_or_b32 s76, s42, s43
; SI-NEXT: s_and_b32 s42, s44, 0xffff
-; SI-NEXT: s_or_b32 s69, s12, s78
-; SI-NEXT: s_and_b32 s12, vcc_hi, 0xffff
-; SI-NEXT: s_or_b32 s77, s42, s99
+; SI-NEXT: s_or_b32 s79, s12, s86
+; SI-NEXT: s_and_b32 s12, s88, 0xffff
+; SI-NEXT: s_or_b32 s77, s42, s85
+; SI-NEXT: s_and_b32 s14, s45, 0xffff
; SI-NEXT: s_or_b32 s42, s12, s4
-; SI-NEXT: s_mov_b32 s12, s45
; SI-NEXT: s_lshr_b64 s[44:45], s[4:5], 16
; SI-NEXT: s_mov_b32 s4, s47
; SI-NEXT: s_lshr_b64 s[46:47], s[6:7], 16
-; SI-NEXT: s_and_b32 s13, s67, 0xffff
-; SI-NEXT: s_and_b32 s14, s70, 0xffff
-; SI-NEXT: s_and_b32 s58, s71, 0xffff
-; SI-NEXT: s_and_b32 vcc_hi, s80, 0xffff
-; SI-NEXT: s_and_b32 s67, s81, 0xffff
+; SI-NEXT: s_and_b32 s13, s89, 0xffff
+; SI-NEXT: s_and_b32 s58, s90, 0xffff
+; SI-NEXT: s_and_b32 s88, vcc_hi, 0xffff
+; SI-NEXT: s_and_b32 s89, s67, 0xffff
; SI-NEXT: s_mov_b32 s47, s4
; SI-NEXT: s_mov_b32 s4, s57
; SI-NEXT: s_lshr_b64 s[56:57], s[8:9], 16
; SI-NEXT: s_mov_b32 s95, s15
-; SI-NEXT: s_and_b32 s70, s83, 0xffff
-; SI-NEXT: s_mov_b32 s45, s12
; SI-NEXT: s_or_b32 s40, s13, s6
; SI-NEXT: s_or_b32 s14, s14, s8
; SI-NEXT: s_mov_b32 s15, s9
@@ -83525,62 +83512,59 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_mov_b32 s13, s11
; SI-NEXT: s_mov_b32 s4, s59
; SI-NEXT: s_lshr_b64 s[58:59], s[10:11], 16
-; SI-NEXT: s_or_b32 s10, vcc_hi, s60
+; SI-NEXT: s_or_b32 s10, s88, s60
; SI-NEXT: s_mov_b32 s11, s61
; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_or_b32 s8, s67, s62
+; SI-NEXT: s_or_b32 s8, s89, s62
; SI-NEXT: s_mov_b32 s9, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
-; SI-NEXT: s_and_b32 s71, s84, 0xffff
+; SI-NEXT: s_and_b32 s90, s69, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s70, 0xffff
; SI-NEXT: s_mov_b32 s59, s4
; SI-NEXT: s_mov_b32 s61, s72
; SI-NEXT: s_mov_b32 s63, s74
-; SI-NEXT: s_or_b32 s6, s70, s76
; SI-NEXT: s_mov_b32 s4, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[76:77], 16
-; SI-NEXT: s_mov_b32 s76, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[68:69], 16
+; SI-NEXT: s_mov_b32 s45, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[78:79], 16
; SI-NEXT: s_mov_b32 s43, s5
; SI-NEXT: s_mov_b32 s41, s7
+; SI-NEXT: s_or_b32 s6, s90, s76
; SI-NEXT: s_mov_b32 s7, s77
; SI-NEXT: s_mov_b32 s73, s4
-; SI-NEXT: s_or_b32 s4, s71, s68
-; SI-NEXT: s_mov_b32 s5, s69
-; SI-NEXT: s_mov_b32 s75, s76
-; SI-NEXT: s_lshr_b32 s71, s85, 16
-; SI-NEXT: s_lshr_b32 s70, s86, 16
-; SI-NEXT: s_lshr_b32 s68, s87, 16
-; SI-NEXT: s_lshr_b32 s69, s96, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s97, 16
-; SI-NEXT: s_lshr_b32 s67, s98, 16
-; SI-NEXT: s_lshr_b32 s80, s99, 16
-; SI-NEXT: s_lshr_b32 s81, s78, 16
+; SI-NEXT: s_or_b32 s4, vcc_hi, s78
+; SI-NEXT: s_mov_b32 s5, s79
+; SI-NEXT: s_mov_b32 s75, s45
+; SI-NEXT: s_lshr_b32 s89, s71, 16
+; SI-NEXT: s_lshr_b32 s88, s80, 16
+; SI-NEXT: s_lshr_b32 s79, s81, 16
+; SI-NEXT: s_lshr_b32 s78, s82, 16
+; SI-NEXT: s_lshr_b32 vcc_hi, s83, 16
+; SI-NEXT: s_lshr_b32 s67, s84, 16
+; SI-NEXT: s_lshr_b32 s90, s85, 16
+; SI-NEXT: s_lshr_b32 s45, s86, 16
; SI-NEXT: s_mov_b64 s[76:77], 0
; SI-NEXT: s_branch .LBB107_3
; SI-NEXT: .LBB107_2:
-; SI-NEXT: s_mov_b32 s79, s5
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_mov_b32 s95, s15
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_mov_b32 s91, s14
-; SI-NEXT: s_mov_b32 s90, s42
-; SI-NEXT: s_mov_b32 s89, s43
-; SI-NEXT: s_mov_b32 s88, s44
; SI-NEXT: s_mov_b64 s[76:77], -1
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr71
+; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr68
+; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $vcc_hi
@@ -83589,10 +83573,10 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr45
; SI-NEXT: .LBB107_3: ; %Flow
; SI-NEXT: s_and_b64 s[76:77], s[76:77], exec
; SI-NEXT: s_cselect_b32 s76, 1, 0
@@ -83601,10 +83585,11 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: v_readlane_b32 s4, v44, 11
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s5, s79, 8
+; SI-NEXT: s_and_b32 s5, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v44, 12
+; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_add_i32 s6, s95, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_lshl_b32 s5, vcc_lo, 24
@@ -83627,16 +83612,19 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s6, s66, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s65, 8
-; SI-NEXT: s_add_i32 s8, s89, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s90, 24
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s8
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s7, s88, 3
+; SI-NEXT: s_and_b32 s7, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 16
+; SI-NEXT: s_lshl_b32 s8, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 15
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: v_readlane_b32 s7, v44, 14
+; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s75, 8
; SI-NEXT: s_add_i32 s9, s73, 3
@@ -83663,15 +83651,16 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s9, s63, 3
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s64, 8
-; SI-NEXT: s_add_i32 s82, s82, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s11, s82, 0xff
; SI-NEXT: s_addk_i32 s9, 0x300
-; SI-NEXT: s_lshl_b32 s10, s55, 24
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_or_b32 s10, s10, s11
-; SI-NEXT: s_or_b32 s9, s10, s9
+; SI-NEXT: s_and_b32 s10, s9, 0xffff
+; SI-NEXT: v_readlane_b32 s9, v44, 13
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: s_lshl_b32 s11, s55, 24
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s9, s11, s9
+; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_add_i32 s10, s50, 3
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s11, s49, 8
@@ -83725,7 +83714,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readlane_b32 s14, v44, 10
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s15, s45, 8
+; SI-NEXT: s_lshl_b32 s15, s68, 8
; SI-NEXT: s_or_b32 s14, s15, s14
; SI-NEXT: s_addk_i32 s14, 0x300
; SI-NEXT: s_and_b32 s15, s14, 0xffff
@@ -83762,49 +83751,50 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_and_b32 s24, s24, 0xffff
; SI-NEXT: s_or_b32 s25, s25, s26
+; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_or_b32 s15, s41, s15
; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s15, s15, s40
; SI-NEXT: s_add_i32 s40, s24, 0x3000000
; SI-NEXT: s_and_b32 s24, s28, 0xff
; SI-NEXT: s_lshl_b32 s25, s29, 8
-; SI-NEXT: s_and_b32 s16, s16, 0xff
-; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s24, 0x300
+; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s19, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s25, s24, 0xffff
-; SI-NEXT: v_readlane_b32 s24, v44, 0
+; SI-NEXT: s_addk_i32 s24, 0x300
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_lshl_b32 s26, s24, 24
-; SI-NEXT: v_readlane_b32 s24, v44, 1
+; SI-NEXT: s_and_b32 s25, s24, 0xffff
+; SI-NEXT: v_readlane_b32 s24, v44, 0
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: s_lshl_b32 s26, s24, 24
+; SI-NEXT: v_readlane_b32 s24, v44, 1
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
; SI-NEXT: s_and_b32 s16, s20, 0xff
; SI-NEXT: s_lshl_b32 s17, s21, 8
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_and_b32 s24, s24, 0xff
+; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s22, 0xff
-; SI-NEXT: s_lshl_b32 s24, s24, 16
+; SI-NEXT: s_and_b32 s24, s24, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s23, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_or_b32 s24, s26, s24
+; SI-NEXT: s_lshl_b32 s24, s24, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_or_b32 s24, s26, s24
; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -83818,7 +83808,6 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
; SI-NEXT: s_add_i32 s41, s24, 0x3000000
-; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[46:47], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[56:57], s[14:15], 16
@@ -83827,14 +83816,14 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshr_b64 s[62:63], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[72:73], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b32 s71, s43, 16
-; SI-NEXT: s_lshr_b32 s70, s41, 16
-; SI-NEXT: s_lshr_b32 s68, s15, 16
-; SI-NEXT: s_lshr_b32 s69, s13, 16
+; SI-NEXT: s_lshr_b32 s89, s43, 16
+; SI-NEXT: s_lshr_b32 s88, s41, 16
+; SI-NEXT: s_lshr_b32 s79, s15, 16
+; SI-NEXT: s_lshr_b32 s78, s13, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s11, 16
; SI-NEXT: s_lshr_b32 s67, s9, 16
-; SI-NEXT: s_lshr_b32 s80, s7, 16
-; SI-NEXT: s_lshr_b32 s81, s5, 16
+; SI-NEXT: s_lshr_b32 s90, s7, 16
+; SI-NEXT: s_lshr_b32 s45, s5, 16
; SI-NEXT: .LBB107_5: ; %end
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
@@ -83843,25 +83832,25 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s17, s44, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s71, 16
+; SI-NEXT: s_lshl_b32 s18, s89, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s40, 0xffff
; SI-NEXT: s_lshl_b32 s19, s46, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s41, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s70, 16
+; SI-NEXT: s_lshl_b32 s20, s88, 16
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_lshl_b32 s20, s56, 16
; SI-NEXT: s_or_b32 s14, s14, s20
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s68, 16
+; SI-NEXT: s_lshl_b32 s20, s79, 16
; SI-NEXT: s_or_b32 s15, s15, s20
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_lshl_b32 s20, s58, 16
; SI-NEXT: s_or_b32 s12, s12, s20
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s69, 16
+; SI-NEXT: s_lshl_b32 s20, s78, 16
; SI-NEXT: s_or_b32 s13, s13, s20
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_lshl_b32 s20, s60, 16
@@ -83879,15 +83868,15 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s20, s72, 16
; SI-NEXT: s_or_b32 s6, s6, s20
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s80, 16
+; SI-NEXT: s_lshl_b32 s20, s90, 16
; SI-NEXT: s_or_b32 s7, s7, s20
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s20, s74, 16
; SI-NEXT: s_or_b32 s4, s4, s20
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s81, 16
+; SI-NEXT: s_lshl_b32 s20, s45, 16
; SI-NEXT: s_or_b32 s5, s5, s20
-; SI-NEXT: v_readlane_b32 s30, v43, 34
+; SI-NEXT: v_readlane_b32 s30, v43, 29
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s17
; SI-NEXT: v_mov_b32_e32 v2, s18
@@ -83904,12 +83893,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_mov_b32_e32 v13, s7
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_readlane_b32 s31, v43, 35
-; SI-NEXT: v_readlane_b32 s99, v43, 33
-; SI-NEXT: v_readlane_b32 s98, v43, 32
-; SI-NEXT: v_readlane_b32 s97, v43, 31
-; SI-NEXT: v_readlane_b32 s96, v43, 30
-; SI-NEXT: v_readlane_b32 s87, v43, 29
+; SI-NEXT: v_readlane_b32 s31, v43, 30
; SI-NEXT: v_readlane_b32 s86, v43, 28
; SI-NEXT: v_readlane_b32 s85, v43, 27
; SI-NEXT: v_readlane_b32 s84, v43, 26
@@ -93918,26 +93902,8 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: v_writelane_b32 v43, s52, 10
; SI-NEXT: v_writelane_b32 v43, s53, 11
; SI-NEXT: v_writelane_b32 v43, s54, 12
-; SI-NEXT: v_writelane_b32 v43, s55, 13
-; SI-NEXT: v_writelane_b32 v43, s64, 14
-; SI-NEXT: v_writelane_b32 v43, s65, 15
-; SI-NEXT: v_writelane_b32 v43, s66, 16
-; SI-NEXT: v_writelane_b32 v43, s67, 17
-; SI-NEXT: v_writelane_b32 v43, s68, 18
-; SI-NEXT: v_writelane_b32 v43, s69, 19
-; SI-NEXT: v_writelane_b32 v43, s70, 20
-; SI-NEXT: v_writelane_b32 v43, s71, 21
-; SI-NEXT: v_writelane_b32 v43, s80, 22
-; SI-NEXT: v_writelane_b32 v43, s81, 23
-; SI-NEXT: v_writelane_b32 v43, s82, 24
-; SI-NEXT: v_writelane_b32 v43, s83, 25
-; SI-NEXT: v_writelane_b32 v43, s84, 26
-; SI-NEXT: v_writelane_b32 v43, s85, 27
-; SI-NEXT: v_writelane_b32 v43, s86, 28
-; SI-NEXT: v_writelane_b32 v43, s87, 29
-; SI-NEXT: v_writelane_b32 v43, s96, 30
-; SI-NEXT: v_writelane_b32 v43, s30, 31
-; SI-NEXT: v_writelane_b32 v43, s31, 32
+; SI-NEXT: v_writelane_b32 v43, s30, 13
+; SI-NEXT: v_writelane_b32 v43, s31, 14
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
@@ -93961,526 +93927,607 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32
+; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: ; implicit-def: $vgpr44 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s9, v1
-; SI-NEXT: v_readfirstlane_b32 s6, v5
-; SI-NEXT: v_readfirstlane_b32 s7, v4
-; SI-NEXT: v_readfirstlane_b32 s8, v3
+; SI-NEXT: v_readfirstlane_b32 s92, v5
+; SI-NEXT: v_writelane_b32 v44, s91, 0
+; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_readfirstlane_b32 s90, v3
+; SI-NEXT: v_readfirstlane_b32 s94, v9
+; SI-NEXT: v_readfirstlane_b32 s38, v8
+; SI-NEXT: v_readfirstlane_b32 s39, v7
+; SI-NEXT: v_readfirstlane_b32 s95, v13
+; SI-NEXT: v_readfirstlane_b32 s36, v12
+; SI-NEXT: v_readfirstlane_b32 s48, v6
+; SI-NEXT: v_readfirstlane_b32 s93, v11
+; SI-NEXT: v_readfirstlane_b32 s45, v17
+; SI-NEXT: v_readfirstlane_b32 s58, v16
+; SI-NEXT: v_readfirstlane_b32 s37, v10
+; SI-NEXT: v_readfirstlane_b32 s59, v15
+; SI-NEXT: v_readfirstlane_b32 s14, v21
+; SI-NEXT: v_readfirstlane_b32 s40, v20
+; SI-NEXT: v_readfirstlane_b32 s60, v14
+; SI-NEXT: v_readfirstlane_b32 s43, v19
+; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: v_readfirstlane_b32 s11, v24
+; SI-NEXT: v_readfirstlane_b32 s44, v18
+; SI-NEXT: v_readfirstlane_b32 s12, v23
+; SI-NEXT: v_readfirstlane_b32 s13, v22
+; SI-NEXT: v_readfirstlane_b32 s7, v28
+; SI-NEXT: v_readfirstlane_b32 s8, v27
; SI-NEXT: v_readfirstlane_b32 s89, v30
-; SI-NEXT: v_readfirstlane_b32 s63, v29
-; SI-NEXT: v_readfirstlane_b32 s72, v28
-; SI-NEXT: v_readfirstlane_b32 s75, v27
-; SI-NEXT: v_readfirstlane_b32 s77, v26
-; SI-NEXT: v_readfirstlane_b32 s59, v25
-; SI-NEXT: v_readfirstlane_b32 s60, v24
-; SI-NEXT: v_readfirstlane_b32 s62, v23
-; SI-NEXT: v_readfirstlane_b32 s73, v22
-; SI-NEXT: v_readfirstlane_b32 s47, v21
-; SI-NEXT: v_readfirstlane_b32 s56, v20
-; SI-NEXT: v_readfirstlane_b32 s58, v19
-; SI-NEXT: v_readfirstlane_b32 s61, v18
-; SI-NEXT: v_readfirstlane_b32 s43, v17
-; SI-NEXT: v_readfirstlane_b32 s44, v16
-; SI-NEXT: v_readfirstlane_b32 s46, v15
-; SI-NEXT: v_readfirstlane_b32 s57, v14
-; SI-NEXT: v_readfirstlane_b32 s15, v13
-; SI-NEXT: v_readfirstlane_b32 s40, v12
-; SI-NEXT: v_readfirstlane_b32 s42, v11
-; SI-NEXT: v_readfirstlane_b32 s45, v10
-; SI-NEXT: v_readfirstlane_b32 s5, v9
-; SI-NEXT: v_readfirstlane_b32 s87, v8
-; SI-NEXT: v_readfirstlane_b32 s14, v7
-; SI-NEXT: v_readfirstlane_b32 s41, v6
-; SI-NEXT: v_readfirstlane_b32 s13, v2
+; SI-NEXT: v_readfirstlane_b32 s6, v29
+; SI-NEXT: v_readfirstlane_b32 s9, v26
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
-; SI-NEXT: s_and_b64 s[10:11], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 s10, v0
-; SI-NEXT: v_writelane_b32 v44, s10, 0
-; SI-NEXT: v_writelane_b32 v44, s9, 1
-; SI-NEXT: v_writelane_b32 v44, s6, 2
-; SI-NEXT: v_writelane_b32 v44, s7, 3
-; SI-NEXT: v_readfirstlane_b32 s55, v32
-; SI-NEXT: v_readfirstlane_b32 s54, v33
-; SI-NEXT: v_readfirstlane_b32 s64, v34
-; SI-NEXT: v_readfirstlane_b32 s65, v35
-; SI-NEXT: v_readfirstlane_b32 s51, v36
+; SI-NEXT: s_and_b64 s[4:5], vcc, exec
+; SI-NEXT: v_readfirstlane_b32 vcc_hi, v0
+; SI-NEXT: v_writelane_b32 v44, vcc_hi, 1
+; SI-NEXT: v_writelane_b32 v44, s92, 2
+; SI-NEXT: v_writelane_b32 v44, s76, 3
+; SI-NEXT: v_writelane_b32 v44, s90, 4
+; SI-NEXT: v_writelane_b32 v44, s94, 5
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v44, s38, 6
+; SI-NEXT: v_writelane_b32 v44, vcc_lo, 7
+; SI-NEXT: v_writelane_b32 v44, s39, 8
+; SI-NEXT: v_writelane_b32 v44, s95, 9
+; SI-NEXT: v_writelane_b32 v44, s36, 10
+; SI-NEXT: v_writelane_b32 v44, s48, 11
+; SI-NEXT: v_writelane_b32 v44, s93, 12
+; SI-NEXT: v_writelane_b32 v44, s45, 13
+; SI-NEXT: v_writelane_b32 v44, s58, 14
+; SI-NEXT: v_writelane_b32 v44, s37, 15
+; SI-NEXT: v_writelane_b32 v44, s59, 16
+; SI-NEXT: v_writelane_b32 v44, s14, 17
+; SI-NEXT: v_writelane_b32 v44, s40, 18
+; SI-NEXT: v_writelane_b32 v44, s60, 19
+; SI-NEXT: v_writelane_b32 v44, s43, 20
+; SI-NEXT: v_writelane_b32 v44, s10, 21
+; SI-NEXT: v_writelane_b32 v44, s11, 22
+; SI-NEXT: v_writelane_b32 v44, s44, 23
+; SI-NEXT: v_writelane_b32 v44, s12, 24
+; SI-NEXT: v_writelane_b32 v44, s13, 25
+; SI-NEXT: v_writelane_b32 v44, s7, 26
+; SI-NEXT: v_readfirstlane_b32 s4, v32
+; SI-NEXT: v_readfirstlane_b32 s5, v33
+; SI-NEXT: v_readfirstlane_b32 s47, v34
+; SI-NEXT: v_readfirstlane_b32 s57, v35
+; SI-NEXT: v_readfirstlane_b32 s46, v36
; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s50, v37
+; SI-NEXT: v_readfirstlane_b32 s61, v37
; SI-NEXT: s_waitcnt vmcnt(12)
-; SI-NEXT: v_readfirstlane_b32 s52, v38
+; SI-NEXT: v_readfirstlane_b32 s41, v38
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s53, v39
+; SI-NEXT: v_readfirstlane_b32 s42, v39
; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_readfirstlane_b32 s39, v48
+; SI-NEXT: v_readfirstlane_b32 s56, v48
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s38, v49
+; SI-NEXT: v_readfirstlane_b32 s62, v49
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s48, v50
+; SI-NEXT: v_readfirstlane_b32 s15, v50
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_readfirstlane_b32 s49, v51
+; SI-NEXT: v_readfirstlane_b32 s63, v51
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s35, v52
+; SI-NEXT: v_readfirstlane_b32 s74, v52
; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_readfirstlane_b32 s34, v53
+; SI-NEXT: v_readfirstlane_b32 s75, v53
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_readfirstlane_b32 s36, v54
+; SI-NEXT: v_readfirstlane_b32 s78, v54
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s37, v55
+; SI-NEXT: v_readfirstlane_b32 s30, v55
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_readfirstlane_b32 s30, v40
+; SI-NEXT: v_readfirstlane_b32 s31, v40
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s4, v41
+; SI-NEXT: v_readfirstlane_b32 s34, v41
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_readfirstlane_b32 s31, v42
-; SI-NEXT: v_writelane_b32 v44, s8, 4
-; SI-NEXT: v_writelane_b32 v44, s5, 5
+; SI-NEXT: v_readfirstlane_b32 s35, v42
+; SI-NEXT: v_writelane_b32 v44, s8, 27
+; SI-NEXT: v_writelane_b32 v44, s6, 28
; SI-NEXT: s_cbranch_scc0 .LBB111_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s11, s16, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s12, s17, 24
-; SI-NEXT: s_or_b32 s74, s12, s11
-; SI-NEXT: s_and_b32 s11, s18, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s12, s19, 24
-; SI-NEXT: s_or_b32 s76, s12, s11
-; SI-NEXT: s_and_b32 s11, s20, 0xff
-; SI-NEXT: s_lshl_b32 s12, s21, 8
-; SI-NEXT: s_or_b32 s12, s11, s12
-; SI-NEXT: s_and_b32 s11, s22, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s78, s23, 24
-; SI-NEXT: s_or_b32 s79, s78, s11
-; SI-NEXT: s_and_b32 s11, s24, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s78, s25, 24
-; SI-NEXT: s_and_b32 s10, s10, 0xff
-; SI-NEXT: s_or_b32 s78, s78, s11
-; SI-NEXT: s_and_b32 s11, s26, 0xff
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_lshl_b32 s9, s9, 24
+; SI-NEXT: s_and_b32 s72, s16, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s73, s17, 24
+; SI-NEXT: s_or_b32 s73, s73, s72
+; SI-NEXT: s_and_b32 s72, s18, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s77, s19, 24
+; SI-NEXT: s_or_b32 s77, s77, s72
+; SI-NEXT: s_and_b32 s72, s20, 0xff
+; SI-NEXT: s_lshl_b32 s79, s21, 8
+; SI-NEXT: s_or_b32 s49, s72, s79
+; SI-NEXT: s_and_b32 s72, s22, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s79, s23, 24
+; SI-NEXT: s_or_b32 s79, s79, s72
+; SI-NEXT: s_and_b32 s72, s24, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s88, s25, 24
+; SI-NEXT: s_or_b32 s72, s88, s72
+; SI-NEXT: s_and_b32 s88, s26, 0xff
+; SI-NEXT: s_and_b32 vcc_lo, vcc_lo, 0xff
+; SI-NEXT: s_and_b32 s76, s76, 0xff
+; SI-NEXT: s_lshl_b32 s88, s88, 16
+; SI-NEXT: s_lshl_b32 s50, s27, 24
+; SI-NEXT: s_lshl_b32 vcc_lo, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s90, s90, 24
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s92, s92, 24
+; SI-NEXT: s_or_b32 s88, s50, s88
+; SI-NEXT: s_and_b32 s50, s28, 0xff
+; SI-NEXT: s_lshl_b32 s51, s29, 8
+; SI-NEXT: s_or_b32 s90, s90, vcc_lo
+; SI-NEXT: s_or_b32 s92, s92, s76
+; SI-NEXT: s_and_b32 s76, s48, 0xff
+; SI-NEXT: s_lshl_b32 vcc_lo, s39, 8
+; SI-NEXT: s_or_b32 s50, s50, s51
+; SI-NEXT: s_or_b32 s51, s76, vcc_lo
+; SI-NEXT: s_and_b32 s76, s38, 0xff
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s94, s94, 24
+; SI-NEXT: s_or_b32 s94, s94, s76
+; SI-NEXT: s_and_b32 s76, s37, 0xff
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s93, s93, 24
+; SI-NEXT: s_or_b32 s93, s93, s76
+; SI-NEXT: s_and_b32 s76, s36, 0xff
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s95, s95, 24
+; SI-NEXT: s_or_b32 s95, s95, s76
+; SI-NEXT: s_mov_b32 s76, s9
+; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_and_b32 s7, s7, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s88, s27, 24
-; SI-NEXT: s_or_b32 s91, s9, s10
-; SI-NEXT: s_and_b32 s9, s13, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_or_b32 s88, s88, s11
-; SI-NEXT: s_and_b32 s11, s28, 0xff
-; SI-NEXT: s_lshl_b32 s90, s29, 8
; SI-NEXT: s_lshl_b32 s9, s9, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_or_b32 s92, s6, s7
-; SI-NEXT: s_and_b32 s6, s41, 0xff
-; SI-NEXT: s_lshl_b32 s7, s14, 8
-; SI-NEXT: s_or_b32 s67, s11, s90
-; SI-NEXT: s_or_b32 s90, s8, s9
-; SI-NEXT: s_or_b32 s8, s6, s7
-; SI-NEXT: s_and_b32 s6, s87, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s5, s5, 24
-; SI-NEXT: s_or_b32 s94, s5, s6
-; SI-NEXT: s_and_b32 s5, s45, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s42, 24
-; SI-NEXT: s_or_b32 s93, s6, s5
-; SI-NEXT: s_and_b32 s5, s40, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s15, 24
-; SI-NEXT: s_or_b32 s95, s6, s5
-; SI-NEXT: s_and_b32 s5, s57, 0xff
-; SI-NEXT: s_lshl_b32 s6, s46, 8
-; SI-NEXT: s_or_b32 s5, s5, s6
-; SI-NEXT: s_and_b32 s6, s44, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s43, 24
-; SI-NEXT: s_or_b32 vcc_hi, s7, s6
-; SI-NEXT: s_and_b32 s6, s61, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s58, 24
-; SI-NEXT: s_or_b32 vcc_lo, s7, s6
-; SI-NEXT: s_and_b32 s6, s56, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s47, 24
-; SI-NEXT: s_or_b32 s86, s7, s6
-; SI-NEXT: s_and_b32 s6, s73, 0xff
-; SI-NEXT: s_lshl_b32 s7, s62, 8
-; SI-NEXT: s_or_b32 s10, s6, s7
-; SI-NEXT: s_and_b32 s6, s60, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s59, 24
-; SI-NEXT: s_or_b32 s85, s7, s6
-; SI-NEXT: s_and_b32 s6, s77, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s75, 24
-; SI-NEXT: s_or_b32 s83, s7, s6
-; SI-NEXT: s_and_b32 s6, s72, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s63, 24
-; SI-NEXT: s_or_b32 s84, s7, s6
-; SI-NEXT: s_and_b32 s6, s89, 0xff
-; SI-NEXT: s_lshl_b32 s7, s31, 8
-; SI-NEXT: s_or_b32 s81, s6, s7
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s7, s30, 24
-; SI-NEXT: s_or_b32 s82, s7, s4
-; SI-NEXT: s_and_b32 s4, s37, 0xff
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s7, s36, 24
-; SI-NEXT: s_or_b32 s71, s7, s4
-; SI-NEXT: s_and_b32 s4, s34, 0xff
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s7, s35, 24
-; SI-NEXT: s_or_b32 s80, s7, s4
-; SI-NEXT: s_and_b32 s4, s49, 0xff
-; SI-NEXT: s_lshl_b32 s7, s48, 8
-; SI-NEXT: s_or_b32 s4, s4, s7
-; SI-NEXT: s_and_b32 s7, s38, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s9, s39, 24
-; SI-NEXT: s_or_b32 s68, s9, s7
-; SI-NEXT: s_and_b32 s7, s53, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s9, s52, 24
-; SI-NEXT: s_or_b32 s7, s9, s7
-; SI-NEXT: s_and_b32 s9, s50, 0xff
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_lshl_b32 s11, s51, 24
-; SI-NEXT: s_or_b32 s66, s11, s9
-; SI-NEXT: s_and_b32 s9, s65, 0xff
-; SI-NEXT: s_lshl_b32 s11, s64, 8
-; SI-NEXT: s_or_b32 s96, s9, s11
-; SI-NEXT: s_and_b32 s9, s54, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_and_b32 s11, s11, 0xff
+; SI-NEXT: s_or_b32 s36, s8, s9
+; SI-NEXT: s_or_b32 s37, s6, s7
+; SI-NEXT: s_and_b32 s6, s89, 0xff
+; SI-NEXT: s_lshl_b32 s8, s35, 8
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_lshl_b32 s10, s10, 24
+; SI-NEXT: s_or_b32 s53, s6, s8
+; SI-NEXT: s_and_b32 s8, s34, 0xff
+; SI-NEXT: s_or_b32 s39, s10, s11
+; SI-NEXT: s_lshl_b32 s9, s8, 16
+; SI-NEXT: s_lshl_b32 s10, s31, 24
+; SI-NEXT: s_mov_b32 s7, s35
+; SI-NEXT: s_or_b32 s35, s10, s9
+; SI-NEXT: s_and_b32 s9, s30, 0xff
+; SI-NEXT: s_and_b32 s40, s40, 0xff
+; SI-NEXT: s_and_b32 s13, s13, 0xff
+; SI-NEXT: s_lshl_b32 s12, s12, 8
; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_lshl_b32 s11, s55, 24
-; SI-NEXT: s_or_b32 s11, s11, s9
-; SI-NEXT: s_lshl_b32 s69, s12, 16
-; SI-NEXT: s_lshl_b32 s70, s67, 16
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_lshl_b32 s9, s5, 16
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_lshl_b32 s81, s81, 16
-; SI-NEXT: s_lshl_b32 s67, s4, 16
-; SI-NEXT: s_lshl_b32 s12, s96, 16
+; SI-NEXT: s_lshl_b32 s11, s78, 24
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_lshl_b32 s14, s14, 24
+; SI-NEXT: s_or_b32 s52, s13, s12
+; SI-NEXT: s_mov_b32 s12, s30
+; SI-NEXT: s_or_b32 s30, s11, s9
+; SI-NEXT: s_and_b32 s11, s75, 0xff
+; SI-NEXT: s_or_b32 s48, s14, s40
+; SI-NEXT: s_lshl_b32 s13, s11, 16
+; SI-NEXT: s_lshl_b32 s14, s74, 24
+; SI-NEXT: s_and_b32 s44, s44, 0xff
+; SI-NEXT: s_mov_b32 s8, s31
+; SI-NEXT: s_or_b32 s31, s14, s13
+; SI-NEXT: s_and_b32 s13, s63, 0xff
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 8
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_lshl_b32 s43, s43, 24
+; SI-NEXT: s_mov_b32 s10, s78
+; SI-NEXT: s_or_b32 s78, s13, s15
+; SI-NEXT: s_and_b32 s15, s62, 0xff
+; SI-NEXT: s_and_b32 vcc_hi, vcc_hi, 0xff
+; SI-NEXT: s_and_b32 s58, s58, 0xff
+; SI-NEXT: s_or_b32 vcc_lo, s43, s44
+; SI-NEXT: s_lshl_b32 s43, s15, 16
+; SI-NEXT: s_lshl_b32 s44, s56, 24
+; SI-NEXT: s_lshl_b32 vcc_hi, vcc_hi, 16
+; SI-NEXT: s_lshl_b32 s91, s91, 24
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_lshl_b32 s45, s45, 24
+; SI-NEXT: s_mov_b32 s15, s56
+; SI-NEXT: s_or_b32 s56, s44, s43
+; SI-NEXT: s_and_b32 s44, s61, 0xff
+; SI-NEXT: s_or_b32 s91, s91, vcc_hi
+; SI-NEXT: s_and_b32 s60, s60, 0xff
+; SI-NEXT: s_lshl_b32 s59, s59, 8
+; SI-NEXT: s_or_b32 vcc_hi, s45, s58
+; SI-NEXT: s_lshl_b32 s58, s44, 16
+; SI-NEXT: s_mov_b32 s44, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 24
+; SI-NEXT: s_or_b32 s38, s60, s59
+; SI-NEXT: s_or_b32 s46, s46, s58
+; SI-NEXT: s_mov_b32 s59, s57
+; SI-NEXT: s_and_b32 s57, s57, 0xff
+; SI-NEXT: s_mov_b32 s58, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 8
+; SI-NEXT: s_mov_b32 s45, s42
+; SI-NEXT: s_and_b32 s42, s42, 0xff
+; SI-NEXT: s_or_b32 s54, s57, s47
+; SI-NEXT: s_mov_b32 s57, s5
+; SI-NEXT: s_and_b32 s5, s5, 0xff
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_mov_b32 s43, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 24
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_mov_b32 s60, s4
+; SI-NEXT: s_lshl_b32 s4, s4, 24
+; SI-NEXT: s_mov_b32 s6, s34
+; SI-NEXT: s_mov_b32 s9, s75
+; SI-NEXT: s_mov_b32 s11, s74
+; SI-NEXT: s_mov_b32 s40, s63
+; SI-NEXT: s_mov_b32 s13, s62
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_mov_b32 s42, s61
+; SI-NEXT: s_or_b32 s75, s4, s5
+; SI-NEXT: s_lshl_b32 s61, s49, 16
+; SI-NEXT: s_lshl_b32 s62, s50, 16
+; SI-NEXT: s_lshl_b32 s63, s51, 16
+; SI-NEXT: s_lshl_b32 s74, s38, 16
+; SI-NEXT: s_lshl_b32 s38, s52, 16
+; SI-NEXT: s_lshl_b32 s34, s53, 16
+; SI-NEXT: s_lshl_b32 s47, s78, 16
+; SI-NEXT: s_lshl_b32 s78, s54, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB111_3
; SI-NEXT: .LBB111_2:
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_mov_b32 s76, s9
+; SI-NEXT: s_mov_b32 s59, s57
+; SI-NEXT: s_mov_b32 s58, s47
+; SI-NEXT: s_mov_b32 s60, s4
+; SI-NEXT: s_mov_b32 s57, s5
+; SI-NEXT: s_mov_b32 s45, s42
+; SI-NEXT: s_mov_b32 s43, s41
+; SI-NEXT: s_mov_b32 s44, s46
+; SI-NEXT: s_mov_b32 s42, s61
+; SI-NEXT: s_mov_b32 s40, s63
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_mov_b32 s15, s56
+; SI-NEXT: s_mov_b32 s13, s62
+; SI-NEXT: s_mov_b32 s12, s30
+; SI-NEXT: s_mov_b32 s10, s78
+; SI-NEXT: s_mov_b32 s11, s74
+; SI-NEXT: s_mov_b32 s9, s75
+; SI-NEXT: s_mov_b32 s7, s35
+; SI-NEXT: s_mov_b32 s8, s31
+; SI-NEXT: s_mov_b32 s6, s34
; SI-NEXT: s_mov_b64 s[4:5], -1
-; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr73
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr61
; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr62
; SI-NEXT: ; implicit-def: $sgpr91
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr92
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr63
; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr93
; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $sgpr9
+; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr86
-; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr85
-; SI-NEXT: ; implicit-def: $sgpr83
-; SI-NEXT: ; implicit-def: $sgpr84
-; SI-NEXT: ; implicit-def: $sgpr81
-; SI-NEXT: ; implicit-def: $sgpr82
-; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr67
-; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr66
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr11
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr47
+; SI-NEXT: ; implicit-def: $sgpr56
+; SI-NEXT: ; implicit-def: $sgpr41
+; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr75
; SI-NEXT: .LBB111_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
-; SI-NEXT: s_mov_b32 s5, s11
-; SI-NEXT: s_mov_b32 s11, s10
-; SI-NEXT: s_mov_b32 s10, s9
+; SI-NEXT: s_mov_b32 s5, s75
+; SI-NEXT: s_mov_b32 s75, s74
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s9, s8
-; SI-NEXT: s_mov_b32 s8, s10
-; SI-NEXT: s_mov_b32 s10, s69
-; SI-NEXT: s_mov_b32 s69, s11
+; SI-NEXT: s_mov_b32 s74, s61
+; SI-NEXT: s_mov_b32 s61, s75
; SI-NEXT: s_cbranch_scc1 .LBB111_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s65, s65, 3
-; SI-NEXT: s_and_b32 s4, s65, 0xff
-; SI-NEXT: s_lshl_b32 s5, s64, 8
-; SI-NEXT: s_add_i32 s54, s54, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s7, s54, 0xff
-; SI-NEXT: s_addk_i32 s4, 0x300
-; SI-NEXT: s_lshl_b32 s5, s55, 24
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_or_b32 s5, s5, s7
-; SI-NEXT: s_add_i32 s53, s53, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s5, s53, 0xff
-; SI-NEXT: s_lshl_b32 s7, s52, 8
-; SI-NEXT: s_add_i32 s50, s50, 3
-; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: s_and_b32 s8, s50, 0xff
-; SI-NEXT: s_addk_i32 s5, 0x300
-; SI-NEXT: s_lshl_b32 s7, s51, 24
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s8
-; SI-NEXT: s_add_i32 s49, s49, 3
-; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: s_and_b32 s7, s49, 0xff
-; SI-NEXT: s_lshl_b32 s8, s48, 8
-; SI-NEXT: s_add_i32 s38, s38, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s9, s38, 0xff
-; SI-NEXT: s_addk_i32 s7, 0x300
-; SI-NEXT: s_lshl_b32 s8, s39, 24
+; SI-NEXT: s_add_i32 s12, s12, 3
+; SI-NEXT: s_and_b32 s12, s12, 0xff
+; SI-NEXT: s_lshl_b32 s10, s10, 8
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_or_b32 s10, s10, s12
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: s_addk_i32 s10, 0x300
+; SI-NEXT: s_lshl_b32 s11, s11, 24
; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_add_i32 s37, s37, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s8, s37, 0xff
-; SI-NEXT: s_lshl_b32 s9, s36, 8
-; SI-NEXT: s_add_i32 s34, s34, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: s_and_b32 s10, s34, 0xff
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_lshl_b32 s9, s35, 24
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_or_b32 s9, s9, s10
-; SI-NEXT: s_or_b32 s8, s9, s8
+; SI-NEXT: s_and_b32 s10, s10, 0xffff
+; SI-NEXT: s_or_b32 s9, s11, s9
; SI-NEXT: s_add_i32 s89, s89, 3
-; SI-NEXT: s_add_i32 s11, s8, 0x3000000
-; SI-NEXT: s_and_b32 s8, s89, 0xff
-; SI-NEXT: s_lshl_b32 s9, s31, 8
+; SI-NEXT: s_or_b32 s9, s9, s10
+; SI-NEXT: s_and_b32 s10, s89, 0xff
+; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
+; SI-NEXT: s_or_b32 s7, s7, s10
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_lshl_b32 s9, s30, 24
+; SI-NEXT: s_addk_i32 s7, 0x300
+; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_or_b32 s6, s9, s6
-; SI-NEXT: s_or_b32 s6, s6, s8
-; SI-NEXT: s_add_i32 s77, s77, 3
-; SI-NEXT: s_add_i32 s12, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s77, 0xff
-; SI-NEXT: s_lshl_b32 s8, s75, 8
-; SI-NEXT: s_add_i32 s72, s72, 3
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s72, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s63, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s73, s73, 3
-; SI-NEXT: s_add_i32 s63, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s73, 0xff
-; SI-NEXT: s_lshl_b32 s8, s62, 8
-; SI-NEXT: s_add_i32 s60, s60, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s60, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s7, s6, 0x3000000
+; SI-NEXT: s_add_i32 s6, s76, 3
+; SI-NEXT: s_and_b32 s8, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 27
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s8
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s59, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s61, s61, 3
-; SI-NEXT: s_add_i32 s59, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s61, 0xff
-; SI-NEXT: s_lshl_b32 s8, s58, 8
-; SI-NEXT: s_add_i32 s56, s56, 3
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s56, 0xff
+; SI-NEXT: s_and_b32 s8, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 28
+; SI-NEXT: s_lshl_b32 s10, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 26
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s10, s6
+; SI-NEXT: s_or_b32 s6, s6, s8
+; SI-NEXT: s_add_i32 s8, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 25
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s10, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 24
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s10
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s47, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s57, s57, 3
-; SI-NEXT: s_add_i32 s47, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s57, 0xff
-; SI-NEXT: s_lshl_b32 s8, s46, 8
-; SI-NEXT: s_add_i32 s44, s44, 3
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s44, 0xff
+; SI-NEXT: s_and_b32 s10, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 21
+; SI-NEXT: s_lshl_b32 s11, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 22
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s11, s6
+; SI-NEXT: s_or_b32 s6, s6, s10
+; SI-NEXT: s_add_i32 s10, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 23
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s11, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 20
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s11
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s43, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s45, s45, 3
-; SI-NEXT: s_add_i32 s8, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s45, 0xff
-; SI-NEXT: s_lshl_b32 s9, s42, 8
+; SI-NEXT: s_and_b32 s11, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 17
+; SI-NEXT: s_lshl_b32 s12, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 18
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s12, s6
+; SI-NEXT: s_or_b32 s6, s6, s11
+; SI-NEXT: s_add_i32 s11, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 19
+; SI-NEXT: s_add_i32 s6, s6, 3
; SI-NEXT: s_add_i32 s40, s40, 3
-; SI-NEXT: s_or_b32 s6, s9, s6
-; SI-NEXT: s_and_b32 s10, s40, 0xff
+; SI-NEXT: s_and_b32 s12, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 16
+; SI-NEXT: s_and_b32 s40, s40, 0xff
+; SI-NEXT: s_lshl_b32 s14, s14, 8
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s14, s14, s40
+; SI-NEXT: s_and_b32 s13, s13, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s12
+; SI-NEXT: s_addk_i32 s14, 0x300
+; SI-NEXT: s_lshl_b32 s15, s15, 24
+; SI-NEXT: s_lshl_b32 s13, s13, 16
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s9, s15, 24
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s9, s9, s10
-; SI-NEXT: s_or_b32 s6, s9, s6
-; SI-NEXT: s_add_i32 s41, s41, 3
-; SI-NEXT: s_add_i32 s15, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s41, 0xff
-; SI-NEXT: s_lshl_b32 s9, s14, 8
-; SI-NEXT: s_or_b32 s6, s9, s6
+; SI-NEXT: s_and_b32 s14, s14, 0xffff
+; SI-NEXT: s_or_b32 s13, s15, s13
+; SI-NEXT: s_and_b32 s12, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 13
+; SI-NEXT: s_or_b32 s13, s13, s14
+; SI-NEXT: s_lshl_b32 s14, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 14
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s14, s6
+; SI-NEXT: s_or_b32 s6, s6, s12
+; SI-NEXT: s_add_i32 s12, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 15
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s14, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 12
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s14
+; SI-NEXT: s_addk_i32 s6, 0x300
+; SI-NEXT: s_and_b32 s14, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 9
+; SI-NEXT: s_lshl_b32 s15, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 10
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s15, s6
+; SI-NEXT: s_or_b32 s6, s6, s14
+; SI-NEXT: s_add_i32 s14, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 11
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s15, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 8
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s15
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_add_i32 s87, s87, 3
-; SI-NEXT: s_and_b32 s9, s6, 0xffff
+; SI-NEXT: s_and_b32 s15, s6, 0xffff
; SI-NEXT: v_readlane_b32 s6, v44, 5
-; SI-NEXT: s_and_b32 s10, s87, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_or_b32 s6, s6, s10
-; SI-NEXT: s_or_b32 s6, s6, s9
-; SI-NEXT: s_add_i32 s9, s6, 0x3000000
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_lshl_b32 s40, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 6
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_add_i32 s4, s59, 3
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s5, s58, 8
+; SI-NEXT: s_add_i32 s41, s57, 3
+; SI-NEXT: s_or_b32 s6, s40, s6
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_and_b32 s41, s41, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s15
+; SI-NEXT: s_addk_i32 s4, 0x300
+; SI-NEXT: s_lshl_b32 s5, s60, 24
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_add_i32 s15, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 7
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_or_b32 s5, s5, s41
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s5, s45, 3
+; SI-NEXT: s_and_b32 s40, s6, 0xff
; SI-NEXT: v_readlane_b32 s6, v44, 4
-; SI-NEXT: s_and_b32 s10, s13, 0xff
+; SI-NEXT: s_and_b32 s5, s5, 0xff
+; SI-NEXT: s_lshl_b32 s41, s43, 8
+; SI-NEXT: s_add_i32 s42, s42, 3
; SI-NEXT: s_lshl_b32 s6, s6, 8
-; SI-NEXT: s_or_b32 s6, s6, s10
+; SI-NEXT: s_or_b32 s5, s41, s5
+; SI-NEXT: s_and_b32 s42, s42, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s40
+; SI-NEXT: s_addk_i32 s5, 0x300
+; SI-NEXT: s_lshl_b32 s41, s44, 24
+; SI-NEXT: s_lshl_b32 s42, s42, 16
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_and_b32 s10, s6, 0xffff
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s40, s6, 0xffff
; SI-NEXT: v_readlane_b32 s6, v44, 2
-; SI-NEXT: s_lshl_b32 s13, s6, 24
+; SI-NEXT: s_or_b32 s5, s41, s5
+; SI-NEXT: s_lshl_b32 s41, s6, 24
; SI-NEXT: v_readlane_b32 s6, v44, 3
; SI-NEXT: s_add_i32 s6, s6, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_or_b32 s6, s13, s6
-; SI-NEXT: s_or_b32 s6, s6, s10
+; SI-NEXT: s_or_b32 s6, s41, s6
+; SI-NEXT: s_or_b32 s6, s6, s40
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s13, s6, 0x3000000
+; SI-NEXT: s_add_i32 s40, s6, 0x3000000
; SI-NEXT: s_and_b32 s6, s28, 0xff
-; SI-NEXT: s_lshl_b32 s10, s29, 8
-; SI-NEXT: s_or_b32 s6, s10, s6
+; SI-NEXT: s_lshl_b32 s28, s29, 8
+; SI-NEXT: s_or_b32 s6, s28, s6
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_and_b32 s10, s6, 0xffff
-; SI-NEXT: v_readlane_b32 s6, v44, 1
-; SI-NEXT: s_lshl_b32 s14, s6, 24
+; SI-NEXT: s_and_b32 s28, s6, 0xffff
; SI-NEXT: v_readlane_b32 s6, v44, 0
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_or_b32 s6, s14, s6
+; SI-NEXT: s_lshl_b32 s29, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 1
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_or_b32 s6, s6, s10
-; SI-NEXT: s_and_b32 s10, s24, 0xff
-; SI-NEXT: s_lshl_b32 s14, s25, 8
-; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_or_b32 s10, s14, s10
-; SI-NEXT: s_and_b32 s24, s26, 0xff
-; SI-NEXT: s_addk_i32 s10, 0x300
-; SI-NEXT: s_lshl_b32 s14, s27, 24
-; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_or_b32 s14, s14, s24
-; SI-NEXT: s_or_b32 s10, s14, s10
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_add_i32 s14, s10, 0x3000000
-; SI-NEXT: s_and_b32 s10, s20, 0xff
-; SI-NEXT: s_lshl_b32 s20, s21, 8
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s24, s24, 0xff
+; SI-NEXT: s_lshl_b32 s25, s25, 8
+; SI-NEXT: s_add_i32 s26, s26, 3
+; SI-NEXT: s_and_b32 s20, s20, 0xff
+; SI-NEXT: s_lshl_b32 s21, s21, 8
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s10, s20, s10
-; SI-NEXT: s_and_b32 s21, s22, 0xff
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_or_b32 s24, s25, s24
+; SI-NEXT: s_and_b32 s26, s26, 0xff
+; SI-NEXT: s_or_b32 s20, s21, s20
+; SI-NEXT: s_and_b32 s22, s22, 0xff
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s10, 0x300
-; SI-NEXT: s_lshl_b32 s20, s23, 24
-; SI-NEXT: s_lshl_b32 s21, s21, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_addk_i32 s24, 0x300
+; SI-NEXT: s_lshl_b32 s25, s27, 24
+; SI-NEXT: s_lshl_b32 s26, s26, 16
+; SI-NEXT: s_addk_i32 s20, 0x300
+; SI-NEXT: s_lshl_b32 s21, s23, 24
+; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s19, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_or_b32 s20, s20, s21
+; SI-NEXT: s_or_b32 s6, s29, s6
+; SI-NEXT: s_and_b32 s24, s24, 0xffff
+; SI-NEXT: s_or_b32 s25, s25, s26
+; SI-NEXT: s_and_b32 s20, s20, 0xffff
+; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_or_b32 s10, s20, s10
+; SI-NEXT: s_or_b32 s6, s6, s28
+; SI-NEXT: s_or_b32 s24, s25, s24
+; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
-; SI-NEXT: s_add_i32 s7, s7, 0x3000000
+; SI-NEXT: s_add_i32 s13, s13, 0x3000000
+; SI-NEXT: s_add_i32 s9, s9, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
-; SI-NEXT: s_add_i32 s10, s10, 0x3000000
+; SI-NEXT: s_add_i32 s24, s24, 0x3000000
+; SI-NEXT: s_add_i32 s20, s20, 0x3000000
; SI-NEXT: s_add_i32 s16, s16, 0x3000000
-; SI-NEXT: s_and_b32 s76, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s74, s16, 16
-; SI-NEXT: s_and_b32 s79, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_and_b32 s88, s14, 0xffff0000
-; SI-NEXT: s_lshl_b32 s78, s14, 16
+; SI-NEXT: s_and_b32 s77, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s16, 16
+; SI-NEXT: s_and_b32 s79, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s20, 16
+; SI-NEXT: s_and_b32 s88, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 s72, s24, 16
; SI-NEXT: s_and_b32 s91, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s70, s6, 16
-; SI-NEXT: s_and_b32 s92, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s13, 16
-; SI-NEXT: s_and_b32 s94, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s95, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s93, s15, 16
-; SI-NEXT: s_and_b32 vcc_hi, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s86, s47, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s47, 16
-; SI-NEXT: s_and_b32 s85, s59, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s59, 16
-; SI-NEXT: s_and_b32 s84, s63, 0xffff0000
-; SI-NEXT: s_lshl_b32 s83, s63, 16
-; SI-NEXT: s_and_b32 s82, s12, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s12, 16
-; SI-NEXT: s_and_b32 s80, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s11, 16
-; SI-NEXT: s_and_b32 s68, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s7, 16
-; SI-NEXT: s_and_b32 s66, s5, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s5, 16
+; SI-NEXT: s_lshl_b32 s62, s6, 16
+; SI-NEXT: s_and_b32 s92, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s90, s40, 16
+; SI-NEXT: s_and_b32 s94, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s15, 16
+; SI-NEXT: s_and_b32 s95, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s14, 16
+; SI-NEXT: s_and_b32 vcc_hi, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s61, s12, 16
+; SI-NEXT: s_and_b32 s48, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_lo, s11, 16
+; SI-NEXT: s_and_b32 s39, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s10, 16
+; SI-NEXT: s_and_b32 s37, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s8, 16
+; SI-NEXT: s_and_b32 s35, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s7, 16
+; SI-NEXT: s_and_b32 s31, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s9, 16
+; SI-NEXT: s_and_b32 s56, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s13, 16
+; SI-NEXT: s_and_b32 s46, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s41, s5, 16
; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s12, s4, 16
+; SI-NEXT: s_lshl_b32 s78, s4, 16
; SI-NEXT: .LBB111_5: ; %end
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s77
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s73
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
; SI-NEXT: v_mul_f32_e64 v1, 1.0, s79
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s74
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
; SI-NEXT: v_mul_f32_e64 v2, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s78
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s72
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
; SI-NEXT: v_mul_f32_e64 v3, 1.0, s91
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s62
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
; SI-NEXT: v_mul_f32_e64 v4, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
@@ -94488,7 +94535,7 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
; SI-NEXT: v_mul_f32_e64 v5, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s63
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
; SI-NEXT: v_mul_f32_e64 v6, 1.0, s95
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
@@ -94496,60 +94543,42 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
; SI-NEXT: v_mul_f32_e64 v7, 1.0, vcc_hi
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s61
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s48
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
; SI-NEXT: v_mul_f32_e64 v8, 1.0, vcc_lo
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s38
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s36
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s34
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s31
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s30
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s56
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s47
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s46
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s41
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
; SI-NEXT: v_mul_f32_e64 v15, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s12
-; SI-NEXT: v_readlane_b32 s30, v43, 31
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s78
+; SI-NEXT: v_readlane_b32 s30, v43, 13
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_readlane_b32 s31, v43, 32
-; SI-NEXT: v_readlane_b32 s96, v43, 30
-; SI-NEXT: v_readlane_b32 s87, v43, 29
-; SI-NEXT: v_readlane_b32 s86, v43, 28
-; SI-NEXT: v_readlane_b32 s85, v43, 27
-; SI-NEXT: v_readlane_b32 s84, v43, 26
-; SI-NEXT: v_readlane_b32 s83, v43, 25
-; SI-NEXT: v_readlane_b32 s82, v43, 24
-; SI-NEXT: v_readlane_b32 s81, v43, 23
-; SI-NEXT: v_readlane_b32 s80, v43, 22
-; SI-NEXT: v_readlane_b32 s71, v43, 21
-; SI-NEXT: v_readlane_b32 s70, v43, 20
-; SI-NEXT: v_readlane_b32 s69, v43, 19
-; SI-NEXT: v_readlane_b32 s68, v43, 18
-; SI-NEXT: v_readlane_b32 s67, v43, 17
-; SI-NEXT: v_readlane_b32 s66, v43, 16
-; SI-NEXT: v_readlane_b32 s65, v43, 15
-; SI-NEXT: v_readlane_b32 s64, v43, 14
-; SI-NEXT: v_readlane_b32 s55, v43, 13
+; SI-NEXT: v_readlane_b32 s31, v43, 14
; SI-NEXT: v_readlane_b32 s54, v43, 12
; SI-NEXT: v_readlane_b32 s53, v43, 11
; SI-NEXT: v_readlane_b32 s52, v43, 10
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
index 1a4c5e9bf06a3c..0a1ee914000643 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
@@ -8355,6 +8355,7 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -8372,16 +8373,14 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -8392,83 +8391,94 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -8478,7 +8488,7 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -8526,11 +8536,11 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -8546,15 +8556,17 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -8593,8 +8605,6 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -8613,6 +8623,7 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -17068,6 +17079,7 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -17085,16 +17097,14 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -17105,83 +17115,94 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -17191,7 +17212,7 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -17239,11 +17260,11 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -17259,15 +17280,17 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -17306,8 +17329,6 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -17326,6 +17347,7 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -24072,6 +24094,7 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -24089,16 +24112,14 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -24109,83 +24130,94 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -24195,7 +24227,7 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -24243,11 +24275,11 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -24263,15 +24295,17 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -24310,8 +24344,6 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -24330,6 +24362,7 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -30350,6 +30383,7 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -30367,16 +30401,14 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -30387,83 +30419,94 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -30473,7 +30516,7 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -30521,11 +30564,11 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -30541,15 +30584,17 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -30588,8 +30633,6 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -30608,6 +30651,7 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
index 35a8e55d252def..afcd7cc0c557d0 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
@@ -8751,8 +8751,9 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -8769,8 +8770,6 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -8791,91 +8790,97 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -8884,7 +8889,7 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -9041,12 +9046,14 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -9095,8 +9102,6 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -9115,6 +9120,7 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -9124,134 +9130,152 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -9261,23 +9285,23 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -9305,15 +9329,15 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -9324,28 +9348,34 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -9384,30 +9414,25 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -18247,8 +18272,9 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -18265,8 +18291,6 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -18287,91 +18311,97 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -18380,7 +18410,7 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -18537,12 +18567,14 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -18591,8 +18623,6 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -18611,6 +18641,7 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -18620,134 +18651,152 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -18757,23 +18806,23 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -18801,15 +18850,15 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -18820,28 +18869,34 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -18880,30 +18935,25 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -25950,8 +26000,9 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -25968,8 +26019,6 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -25990,91 +26039,97 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -26083,7 +26138,7 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -26240,12 +26295,14 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -26294,8 +26351,6 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -26314,6 +26369,7 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -26323,134 +26379,152 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -26460,23 +26534,23 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -26504,15 +26578,15 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -26523,28 +26597,34 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -26583,30 +26663,25 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -32901,8 +32976,9 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -32919,8 +32995,6 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -32941,91 +33015,97 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -33034,7 +33114,7 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -33191,12 +33271,14 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -33245,8 +33327,6 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -33265,6 +33345,7 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -33274,134 +33355,152 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -33411,23 +33510,23 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -33455,15 +33554,15 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -33474,28 +33573,34 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -33534,30 +33639,25 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
index ceb81e46a53789..17afa1ed8fcc60 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
@@ -9455,40 +9455,44 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -9497,103 +9501,112 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -9602,9 +9615,9 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -9614,7 +9627,7 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -9624,14 +9637,14 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -9740,10 +9753,11 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -9751,12 +9765,14 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -9764,10 +9780,11 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -9775,12 +9792,14 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -9829,30 +9848,25 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -9862,148 +9876,172 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -10013,35 +10051,35 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -10053,25 +10091,28 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -10080,32 +10121,39 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -10144,34 +10192,27 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -19703,40 +19744,44 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -19745,103 +19790,112 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -19850,9 +19904,9 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -19862,7 +19916,7 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -19872,14 +19926,14 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -19988,10 +20042,11 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -19999,12 +20054,14 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -20012,10 +20069,11 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -20023,12 +20081,14 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -20077,30 +20137,25 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -20110,148 +20165,172 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -20261,35 +20340,35 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -20301,25 +20380,28 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -20328,32 +20410,39 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -20392,34 +20481,27 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -28076,40 +28158,44 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -28118,103 +28204,112 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -28223,9 +28318,9 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -28235,7 +28330,7 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -28245,14 +28340,14 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -28361,10 +28456,11 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -28372,12 +28468,14 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -28385,10 +28483,11 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -28396,12 +28495,14 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -28450,30 +28551,25 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -28483,148 +28579,172 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -28634,35 +28754,35 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -28674,25 +28794,28 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -28701,32 +28824,39 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -28765,34 +28895,27 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -35670,40 +35793,44 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -35712,103 +35839,112 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -35817,9 +35953,9 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -35829,7 +35965,7 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -35839,14 +35975,14 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -35955,10 +36091,11 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -35966,12 +36103,14 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -35979,10 +36118,11 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -35990,12 +36130,14 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -36044,30 +36186,25 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -36077,148 +36214,172 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -36228,35 +36389,35 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -36268,25 +36429,28 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -36295,32 +36459,39 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -36359,34 +36530,27 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -37822,9 +37986,9 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v25, s56, 5
; SI-NEXT: s_lshr_b32 s41, s45, 16
; SI-NEXT: v_writelane_b32 v25, s47, 6
-; SI-NEXT: v_readfirstlane_b32 s48, v6
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v6
; SI-NEXT: v_writelane_b32 v25, s41, 7
-; SI-NEXT: s_lshr_b32 s15, s48, 16
+; SI-NEXT: s_lshr_b32 s15, vcc_lo, 16
; SI-NEXT: v_writelane_b32 v25, s45, 8
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v25, s15, 9
@@ -37832,8 +37996,8 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v25, s43, 10
; SI-NEXT: v_readfirstlane_b32 s11, v5
; SI-NEXT: v_writelane_b32 v25, s13, 11
-; SI-NEXT: v_readfirstlane_b32 s5, v9
-; SI-NEXT: v_readfirstlane_b32 s37, v8
+; SI-NEXT: v_readfirstlane_b32 s7, v9
+; SI-NEXT: v_readfirstlane_b32 s30, v8
; SI-NEXT: v_readfirstlane_b32 s9, v7
; SI-NEXT: v_writelane_b32 v25, s11, 12
; SI-NEXT: s_lshr_b32 s89, s29, 16
@@ -37847,17 +38011,17 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s50, s18, 16
; SI-NEXT: s_lshr_b32 s61, s17, 16
; SI-NEXT: s_lshr_b32 s36, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s5, 16
-; SI-NEXT: s_lshr_b32 s7, s37, 16
+; SI-NEXT: s_lshr_b32 s31, s7, 16
+; SI-NEXT: s_lshr_b32 s5, s30, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s9, 16
; SI-NEXT: s_lshr_b32 s51, s11, 16
-; SI-NEXT: s_lshr_b32 s49, s13, 16
-; SI-NEXT: s_lshr_b32 s91, s43, 16
+; SI-NEXT: s_lshr_b32 s93, s13, 16
+; SI-NEXT: s_lshr_b32 s37, s43, 16
; SI-NEXT: v_readfirstlane_b32 s10, v10
; SI-NEXT: v_writelane_b32 v25, s9, 13
; SI-NEXT: s_cmp_lg_u32 s10, 0
-; SI-NEXT: v_writelane_b32 v25, s5, 14
-; SI-NEXT: v_writelane_b32 v25, s7, 15
+; SI-NEXT: v_writelane_b32 v25, s7, 14
+; SI-NEXT: v_writelane_b32 v25, s5, 15
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s10, s16, 0xffff
@@ -37904,28 +38068,26 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s73, s45, s47
; SI-NEXT: s_and_b32 s45, s23, 0xffff
; SI-NEXT: s_lshl_b32 s47, s75, 16
-; SI-NEXT: s_mov_b32 vcc_lo, s75
+; SI-NEXT: s_mov_b32 s34, s75
; SI-NEXT: s_or_b32 s75, s45, s47
; SI-NEXT: s_and_b32 s45, s25, 0xffff
; SI-NEXT: s_lshl_b32 s47, s77, 16
-; SI-NEXT: s_mov_b32 s30, s77
+; SI-NEXT: s_mov_b32 s35, s77
; SI-NEXT: s_or_b32 s77, s45, s47
; SI-NEXT: s_and_b32 s45, s27, 0xffff
; SI-NEXT: s_lshl_b32 s47, s79, 16
-; SI-NEXT: s_mov_b32 s34, s79
+; SI-NEXT: s_mov_b32 s48, s79
; SI-NEXT: s_or_b32 s79, s45, s47
; SI-NEXT: s_and_b32 s45, s29, 0xffff
; SI-NEXT: s_lshl_b32 s47, s89, 16
-; SI-NEXT: s_mov_b32 s35, s89
+; SI-NEXT: s_mov_b32 s49, s89
; SI-NEXT: s_or_b32 s89, s45, s47
; SI-NEXT: s_and_b32 s43, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s45, s91, 16
-; SI-NEXT: s_mov_b32 s95, s37
-; SI-NEXT: s_mov_b32 s37, s48
-; SI-NEXT: s_mov_b32 s48, s91
+; SI-NEXT: s_lshl_b32 s45, s37, 16
; SI-NEXT: s_or_b32 s91, s43, s45
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s49, 16
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_mov_b32 s95, s93
; SI-NEXT: s_or_b32 s93, s13, s43
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s13, s51, 16
@@ -37933,48 +38095,51 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s11, vcc_hi, 16
; SI-NEXT: s_or_b32 s57, s9, s11
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_lshl_b32 s9, s31, 16
-; SI-NEXT: s_or_b32 s53, s5, s9
+; SI-NEXT: s_mov_b32 s47, s61
+; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
+; SI-NEXT: s_or_b32 s53, s7, s9
+; SI-NEXT: v_writelane_b32 v25, s60, 16
; SI-NEXT: s_mov_b32 s45, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
; SI-NEXT: s_mov_b32 s43, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
-; SI-NEXT: s_mov_b32 s9, s91
-; SI-NEXT: s_lshr_b64 s[90:91], s[90:91], 16
-; SI-NEXT: s_mov_b32 s47, s61
-; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_mov_b32 s63, s41
+; SI-NEXT: s_mov_b32 s13, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
+; SI-NEXT: s_mov_b32 s7, s93
+; SI-NEXT: s_lshr_b64 s[92:93], s[92:93], 16
+; SI-NEXT: s_or_b32 s4, s4, s58
+; SI-NEXT: s_lshl_b32 s52, s5, 16
+; SI-NEXT: v_writelane_b32 v25, s61, 17
; SI-NEXT: s_mov_b32 s73, s94
-; SI-NEXT: s_mov_b32 s41, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
-; SI-NEXT: s_mov_b32 s91, s48
-; SI-NEXT: s_mov_b32 s48, s37
-; SI-NEXT: s_mov_b32 s37, s95
+; SI-NEXT: s_mov_b32 s79, s48
+; SI-NEXT: s_mov_b32 s48, s62
+; SI-NEXT: s_mov_b32 s93, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[58:59], 16
-; SI-NEXT: s_or_b32 s4, s4, s58
-; SI-NEXT: s_lshl_b32 s52, s7, 16
+; SI-NEXT: s_mov_b32 s60, vcc_lo
+; SI-NEXT: s_and_b32 s58, vcc_lo, 0xffff
+; SI-NEXT: s_mov_b32 s62, vcc_hi
+; SI-NEXT: s_lshr_b64 vcc, s[56:57], 16
; SI-NEXT: s_mov_b32 s61, s15
-; SI-NEXT: s_mov_b32 s75, vcc_lo
+; SI-NEXT: s_mov_b32 s63, s41
+; SI-NEXT: s_mov_b32 s41, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
; SI-NEXT: s_mov_b32 s15, s77
; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
-; SI-NEXT: s_and_b32 s58, s48, 0xffff
-; SI-NEXT: s_mov_b32 s95, vcc_hi
-; SI-NEXT: s_lshr_b64 vcc, s[56:57], 16
-; SI-NEXT: s_mov_b32 s77, s30
-; SI-NEXT: s_mov_b32 s13, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
; SI-NEXT: s_mov_b32 s11, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[88:89], 16
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 vcc_hi, s95
-; SI-NEXT: s_and_b32 s56, s37, 0xffff
+; SI-NEXT: s_mov_b32 vcc_hi, s62
+; SI-NEXT: s_mov_b32 s62, s30
+; SI-NEXT: s_and_b32 s56, s30, 0xffff
; SI-NEXT: s_mov_b32 s95, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[52:53], 16
-; SI-NEXT: s_mov_b32 s79, s34
-; SI-NEXT: s_mov_b32 s89, s35
-; SI-NEXT: s_mov_b32 s7, s93
-; SI-NEXT: s_lshr_b64 s[92:93], s[92:93], 16
+; SI-NEXT: s_mov_b32 s75, s34
+; SI-NEXT: s_mov_b32 s77, s35
+; SI-NEXT: s_mov_b32 s89, s49
+; SI-NEXT: s_mov_b32 s9, s91
+; SI-NEXT: s_lshr_b64 s[90:91], s[90:91], 16
; SI-NEXT: s_mov_b32 s5, s59
; SI-NEXT: s_mov_b32 s59, s57
; SI-NEXT: s_or_b32 s56, s56, s52
@@ -37983,11 +38148,15 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_mov_b64 s[34:35], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s62, s30
+; SI-NEXT: s_mov_b32 s60, vcc_lo
; SI-NEXT: s_mov_b64 s[34:35], -1
+; SI-NEXT: v_writelane_b32 v25, s4, 16
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr60
+; SI-NEXT: v_writelane_b32 v25, s5, 17
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; implicit-def: $sgpr48
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr40
@@ -38012,9 +38181,11 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_and_b64 s[34:35], s[34:35], exec
; SI-NEXT: s_cselect_b32 s95, 1, 0
; SI-NEXT: s_cmp_lg_u32 s95, 1
+; SI-NEXT: v_readlane_b32 s34, v25, 16
+; SI-NEXT: v_readlane_b32 s35, v25, 17
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s4, s37, 3
+; SI-NEXT: s_add_i32 s4, s62, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v25, 15
; SI-NEXT: s_lshl_b32 s4, s4, 16
@@ -38026,7 +38197,7 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s48, 3
+; SI-NEXT: s_add_i32 s4, s60, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v25, 9
; SI-NEXT: s_lshl_b32 s4, s4, 16
@@ -38058,7 +38229,7 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_readlane_b32 s7, v25, 11
; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s49, 16
+; SI-NEXT: s_lshl_b32 s8, s93, 16
; SI-NEXT: s_or_b32 s7, s8, s7
; SI-NEXT: v_readlane_b32 s8, v25, 4
; SI-NEXT: s_add_i32 s8, s8, 3
@@ -38069,7 +38240,7 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_readlane_b32 s9, v25, 10
; SI-NEXT: s_add_i32 s9, s9, 3
; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s10, s91, 16
+; SI-NEXT: s_lshl_b32 s10, s37, 16
; SI-NEXT: s_or_b32 s9, s10, s9
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: v_readlane_b32 s10, v25, 2
@@ -38136,31 +38307,31 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s46, s16, 0x30000
; SI-NEXT: s_and_b32 s16, s17, 0xffff
; SI-NEXT: s_lshl_b32 s17, s61, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s8, s8, 0x30000
-; SI-NEXT: s_add_i32 s9, s9, 0x30000
+; SI-NEXT: s_add_i32 s6, s6, 0x30000
+; SI-NEXT: s_add_i32 s7, s7, 0x30000
; SI-NEXT: s_add_i32 s10, s10, 0x30000
; SI-NEXT: s_add_i32 s11, s11, 0x30000
; SI-NEXT: s_add_i32 s12, s12, 0x30000
; SI-NEXT: s_add_i32 s13, s13, 0x30000
; SI-NEXT: s_add_i32 s14, s14, 0x30000
; SI-NEXT: s_add_i32 s15, s15, 0x30000
-; SI-NEXT: s_add_i32 s47, s16, 0x30000
+; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_add_i32 s6, s6, 0x30000
-; SI-NEXT: s_add_i32 s7, s7, 0x30000
-; SI-NEXT: s_lshr_b64 s[60:61], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[44:45], 16
+; SI-NEXT: s_add_i32 s8, s8, 0x30000
+; SI-NEXT: s_add_i32 s9, s9, 0x30000
+; SI-NEXT: s_add_i32 s47, s16, 0x30000
; SI-NEXT: s_lshr_b64 s[72:73], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[76:77], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[10:11], 16
-; SI-NEXT: s_lshr_b64 s[90:91], s[8:9], 16
+; SI-NEXT: s_lshr_b64 s[92:93], s[6:7], 16
; SI-NEXT: s_lshr_b64 vcc, s[58:59], 16
; SI-NEXT: s_lshr_b64 s[30:31], s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[6:7], 16
+; SI-NEXT: s_lshr_b64 s[34:35], s[46:47], 16
+; SI-NEXT: s_lshr_b64 s[48:49], s[44:45], 16
+; SI-NEXT: s_lshr_b64 s[90:91], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[94:95], s[4:5], 16
; SI-NEXT: s_lshr_b32 s61, s47, 16
; SI-NEXT: s_lshr_b32 s63, s45, 16
@@ -38169,20 +38340,20 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s77, s15, 16
; SI-NEXT: s_lshr_b32 s79, s13, 16
; SI-NEXT: s_lshr_b32 s89, s11, 16
-; SI-NEXT: s_lshr_b32 s91, s9, 16
-; SI-NEXT: s_lshr_b32 s49, s7, 16
+; SI-NEXT: s_lshr_b32 s37, s9, 16
+; SI-NEXT: s_lshr_b32 s93, s7, 16
; SI-NEXT: s_lshr_b32 s51, s5, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s59, 16
; SI-NEXT: s_lshr_b32 s31, s57, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s16, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s60, 16
+; SI-NEXT: s_lshl_b32 s17, s34, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s47, 0xffff
; SI-NEXT: s_lshl_b32 s18, s61, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s62, 16
+; SI-NEXT: s_lshl_b32 s19, s48, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s45, 0xffff
; SI-NEXT: s_lshl_b32 s20, s63, 16
@@ -38221,13 +38392,13 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s24, s90, 16
; SI-NEXT: s_or_b32 s8, s8, s24
; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s91, 16
+; SI-NEXT: s_lshl_b32 s24, s37, 16
; SI-NEXT: s_or_b32 s9, s9, s24
; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_lshl_b32 s24, s92, 16
; SI-NEXT: s_or_b32 s6, s6, s24
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s49, 16
+; SI-NEXT: s_lshl_b32 s24, s93, 16
; SI-NEXT: s_or_b32 s7, s7, s24
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s24, s94, 16
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
index 8d46f9af9325a4..868ed4dcae84e4 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
@@ -6587,6 +6587,7 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -6606,145 +6607,189 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB15_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -6752,43 +6797,47 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_cbranch_scc1 .LBB15_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -6798,27 +6847,27 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -6828,61 +6877,66 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB15_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -6909,17 +6963,7 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -6940,6 +6984,7 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -10169,159 +10214,182 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -10330,9 +10398,9 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -10344,7 +10412,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -10352,11 +10420,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -10367,7 +10435,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -10376,11 +10444,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -10391,7 +10459,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -10450,9 +10518,10 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -10460,10 +10529,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -10471,12 +10541,14 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -10484,10 +10556,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -10495,12 +10568,14 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -10508,10 +10583,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -10519,12 +10595,14 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -10573,34 +10651,27 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -10610,6 +10681,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -10629,143 +10701,172 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -10775,39 +10876,39 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -10815,67 +10916,81 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -10914,18 +11029,8 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -10946,6 +11051,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -17344,6 +17450,7 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -17363,145 +17470,189 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB31_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB31_3
; VI-NEXT: .LBB31_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB31_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -17509,43 +17660,47 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB31_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -17555,27 +17710,27 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -17585,61 +17740,66 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB31_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -17666,17 +17826,7 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -17697,6 +17847,7 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -21245,159 +21396,182 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -21406,9 +21580,9 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -21420,7 +21594,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -21428,11 +21602,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -21443,7 +21617,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -21452,11 +21626,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -21467,7 +21641,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -21526,9 +21700,10 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -21536,10 +21711,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -21547,12 +21723,14 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -21560,10 +21738,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -21571,12 +21750,14 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -21584,10 +21765,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -21595,12 +21777,14 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -21649,34 +21833,27 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -21686,6 +21863,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -21705,143 +21883,172 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -21851,39 +22058,39 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -21891,67 +22098,81 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -21990,18 +22211,8 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -22022,6 +22233,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -26710,6 +26922,7 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -26729,145 +26942,189 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB43_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB43_3
; VI-NEXT: .LBB43_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB43_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -26875,43 +27132,47 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_cbranch_scc1 .LBB43_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -26921,27 +27182,27 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -26951,61 +27212,66 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB43_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -27032,17 +27298,7 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -27063,6 +27319,7 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -30322,159 +30579,182 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -30483,9 +30763,9 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -30497,7 +30777,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -30505,11 +30785,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -30520,7 +30800,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -30529,11 +30809,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -30544,7 +30824,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -30603,9 +30883,10 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -30613,10 +30894,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -30624,12 +30906,14 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -30637,10 +30921,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -30648,12 +30933,14 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -30661,10 +30948,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -30672,12 +30960,14 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -30726,34 +31016,27 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -30763,6 +31046,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -30782,143 +31066,172 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -30928,39 +31241,39 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -30968,67 +31281,81 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -31067,18 +31394,8 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -31099,6 +31416,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -34840,6 +35158,7 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -34859,145 +35178,189 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB51_3
; VI-NEXT: .LBB51_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB51_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -35005,43 +35368,47 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB51_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -35051,27 +35418,27 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -35081,61 +35448,66 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB51_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -35162,17 +35534,7 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -35193,6 +35555,7 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -38640,159 +39003,182 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -38801,9 +39187,9 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -38815,7 +39201,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -38823,11 +39209,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -38838,7 +39224,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -38847,11 +39233,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -38862,7 +39248,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -38921,9 +39307,10 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -38931,10 +39318,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -38942,12 +39330,14 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -38955,10 +39345,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -38966,12 +39357,14 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -38979,10 +39372,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -38990,12 +39384,14 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -39044,34 +39440,27 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -39081,6 +39470,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -39100,143 +39490,172 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -39246,39 +39665,39 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -39286,67 +39705,81 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -39385,18 +39818,8 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -39417,6 +39840,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -41012,10 +41436,8 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v26, s51, 9
; SI-NEXT: v_writelane_b32 v26, s52, 10
; SI-NEXT: v_writelane_b32 v26, s53, 11
-; SI-NEXT: v_writelane_b32 v26, s54, 12
-; SI-NEXT: v_writelane_b32 v26, s55, 13
-; SI-NEXT: v_writelane_b32 v26, s30, 14
-; SI-NEXT: v_writelane_b32 v26, s31, 15
+; SI-NEXT: v_writelane_b32 v26, s30, 12
+; SI-NEXT: v_writelane_b32 v26, s31, 13
; SI-NEXT: s_lshr_b32 s40, s16, 16
; SI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
; SI-NEXT: s_lshr_b32 s14, s18, 16
@@ -41030,21 +41452,21 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s6, s26, 16
; SI-NEXT: v_writelane_b32 v27, s8, 4
; SI-NEXT: s_lshr_b32 s4, s28, 16
-; SI-NEXT: v_readfirstlane_b32 s48, v0
+; SI-NEXT: v_readfirstlane_b32 s92, v0
; SI-NEXT: v_writelane_b32 v27, s6, 5
-; SI-NEXT: v_readfirstlane_b32 s49, v2
-; SI-NEXT: s_lshr_b32 s46, s48, 16
+; SI-NEXT: v_readfirstlane_b32 s94, v2
+; SI-NEXT: s_lshr_b32 s46, s92, 16
; SI-NEXT: v_writelane_b32 v27, s4, 6
-; SI-NEXT: v_readfirstlane_b32 s50, v4
-; SI-NEXT: s_lshr_b32 s45, s49, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
+; SI-NEXT: s_lshr_b32 s45, s94, 16
; SI-NEXT: v_writelane_b32 v27, s46, 7
-; SI-NEXT: v_readfirstlane_b32 s51, v6
-; SI-NEXT: s_lshr_b32 s44, s50, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v6
+; SI-NEXT: s_lshr_b32 s44, vcc_lo, 16
; SI-NEXT: v_writelane_b32 v27, s45, 8
-; SI-NEXT: v_readfirstlane_b32 s52, v8
-; SI-NEXT: s_lshr_b32 s42, s51, 16
+; SI-NEXT: v_readfirstlane_b32 s34, v8
+; SI-NEXT: s_lshr_b32 s42, s30, 16
; SI-NEXT: v_writelane_b32 v27, s44, 9
-; SI-NEXT: s_lshr_b32 s15, s52, 16
+; SI-NEXT: s_lshr_b32 s15, s34, 16
; SI-NEXT: v_writelane_b32 v27, s42, 10
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v27, s15, 11
@@ -41054,10 +41476,11 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v27, s41, 13
; SI-NEXT: v_readfirstlane_b32 s11, v7
; SI-NEXT: v_writelane_b32 v27, s13, 14
-; SI-NEXT: v_readfirstlane_b32 s7, v11
-; SI-NEXT: v_readfirstlane_b32 s53, v10
; SI-NEXT: v_readfirstlane_b32 s9, v9
; SI-NEXT: v_writelane_b32 v27, s11, 15
+; SI-NEXT: v_readfirstlane_b32 s5, v11
+; SI-NEXT: v_readfirstlane_b32 s36, v10
+; SI-NEXT: v_writelane_b32 v27, s9, 16
; SI-NEXT: s_lshr_b32 s91, s29, 16
; SI-NEXT: s_lshr_b32 s89, s27, 16
; SI-NEXT: s_lshr_b32 s79, s25, 16
@@ -41065,18 +41488,18 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s75, s21, 16
; SI-NEXT: s_lshr_b32 s73, s19, 16
; SI-NEXT: s_lshr_b32 s63, s17, 16
-; SI-NEXT: s_lshr_b32 s37, s7, 16
-; SI-NEXT: s_lshr_b32 s5, s53, 16
+; SI-NEXT: s_lshr_b32 s37, s5, 16
+; SI-NEXT: s_lshr_b32 s7, s36, 16
; SI-NEXT: s_lshr_b32 s35, s9, 16
; SI-NEXT: s_lshr_b32 s31, s11, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s13, 16
; SI-NEXT: s_lshr_b32 s95, s41, 16
; SI-NEXT: s_lshr_b32 s93, s43, 16
; SI-NEXT: v_readfirstlane_b32 s47, v12
-; SI-NEXT: v_writelane_b32 v27, s9, 16
+; SI-NEXT: v_writelane_b32 v27, s5, 17
; SI-NEXT: s_cmp_lg_u32 s47, 0
-; SI-NEXT: v_writelane_b32 v27, s7, 17
-; SI-NEXT: v_writelane_b32 v27, s5, 18
+; SI-NEXT: v_writelane_b32 v27, s92, 18
+; SI-NEXT: v_writelane_b32 v27, s7, 19
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s47, s16, 0xffff
@@ -41115,19 +41538,19 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s73, s45, s47
; SI-NEXT: s_and_b32 s45, s21, 0xffff
; SI-NEXT: s_lshl_b32 s47, s75, 16
-; SI-NEXT: s_mov_b32 s92, s75
+; SI-NEXT: s_mov_b32 s38, s75
; SI-NEXT: s_or_b32 s75, s45, s47
; SI-NEXT: s_and_b32 s45, s23, 0xffff
; SI-NEXT: s_lshl_b32 s47, s77, 16
-; SI-NEXT: s_mov_b32 s94, s77
+; SI-NEXT: s_mov_b32 s39, s77
; SI-NEXT: s_or_b32 s77, s45, s47
; SI-NEXT: s_and_b32 s45, s25, 0xffff
; SI-NEXT: s_lshl_b32 s47, s79, 16
-; SI-NEXT: s_mov_b32 vcc_lo, s79
+; SI-NEXT: s_mov_b32 s48, s79
; SI-NEXT: s_or_b32 s79, s45, s47
; SI-NEXT: s_and_b32 s45, s27, 0xffff
; SI-NEXT: s_lshl_b32 s47, s89, 16
-; SI-NEXT: s_mov_b32 s30, s89
+; SI-NEXT: s_mov_b32 s49, s89
; SI-NEXT: s_or_b32 s89, s45, s47
; SI-NEXT: s_and_b32 s45, s29, 0xffff
; SI-NEXT: s_lshl_b32 s47, s91, 16
@@ -41141,82 +41564,96 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
; SI-NEXT: s_or_b32 s47, s13, s41
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s31, 16
; SI-NEXT: s_mov_b32 s41, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
+; SI-NEXT: v_writelane_b32 v27, s62, 20
+; SI-NEXT: v_writelane_b32 v27, s63, 21
+; SI-NEXT: s_mov_b32 s63, s15
+; SI-NEXT: s_mov_b32 s15, s73
+; SI-NEXT: s_lshr_b64 s[50:51], s[72:73], 16
+; SI-NEXT: s_mov_b32 s73, s90
+; SI-NEXT: s_mov_b32 s62, s91
+; SI-NEXT: s_lshr_b64 s[90:91], s[60:61], 16
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s31, 16
+; SI-NEXT: s_mov_b32 s91, s62
+; SI-NEXT: s_and_b32 s60, s92, 0xffff
+; SI-NEXT: s_mov_b32 s62, s93
+; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
; SI-NEXT: s_or_b32 s45, s11, s13
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s11, s35, 16
-; SI-NEXT: s_mov_b32 s63, s15
-; SI-NEXT: s_mov_b32 s15, s73
-; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
; SI-NEXT: s_mov_b32 s13, s75
; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
+; SI-NEXT: s_or_b32 s60, s60, s58
+; SI-NEXT: s_mov_b32 s93, s62
+; SI-NEXT: s_mov_b32 s62, s94
+; SI-NEXT: s_and_b32 s58, s94, 0xffff
+; SI-NEXT: s_mov_b32 s72, s95
+; SI-NEXT: s_lshr_b64 s[94:95], s[56:57], 16
; SI-NEXT: s_or_b32 s43, s9, s11
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_lshl_b32 s9, s37, 16
-; SI-NEXT: s_mov_b32 s73, s90
-; SI-NEXT: s_mov_b32 s75, s92
-; SI-NEXT: s_mov_b32 s11, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
-; SI-NEXT: s_mov_b32 s92, s91
-; SI-NEXT: s_lshr_b64 s[90:91], s[60:61], 16
-; SI-NEXT: s_or_b32 s55, s7, s9
-; SI-NEXT: s_mov_b32 s77, s94
-; SI-NEXT: s_mov_b32 s9, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
-; SI-NEXT: s_mov_b32 s91, s92
-; SI-NEXT: s_and_b32 s60, s48, 0xffff
-; SI-NEXT: s_mov_b32 s94, s93
-; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
-; SI-NEXT: s_mov_b32 s79, vcc_lo
+; SI-NEXT: s_lshl_b32 s52, s7, 16
+; SI-NEXT: s_mov_b32 s75, s38
; SI-NEXT: s_mov_b32 s7, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[88:89], 16
-; SI-NEXT: s_or_b32 s60, s60, s58
-; SI-NEXT: s_mov_b32 s93, s94
-; SI-NEXT: s_and_b32 s58, s49, 0xffff
-; SI-NEXT: s_mov_b32 vcc_lo, s95
-; SI-NEXT: s_lshr_b64 s[94:95], s[56:57], 16
-; SI-NEXT: s_mov_b32 s89, s30
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 s95, vcc_lo
-; SI-NEXT: s_and_b32 s56, s50, 0xffff
-; SI-NEXT: s_mov_b32 s30, vcc_hi
+; SI-NEXT: s_mov_b32 s95, s72
+; SI-NEXT: s_mov_b32 s72, vcc_lo
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_mov_b32 s38, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[46:47], 16
+; SI-NEXT: s_or_b32 s53, s5, s9
+; SI-NEXT: s_mov_b32 s9, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
+; SI-NEXT: s_mov_b32 s89, s49
; SI-NEXT: s_or_b32 s56, s56, s46
-; SI-NEXT: s_mov_b32 vcc_hi, s30
-; SI-NEXT: s_and_b32 s46, s51, 0xffff
-; SI-NEXT: s_mov_b32 s34, s31
+; SI-NEXT: s_mov_b32 vcc_hi, s38
+; SI-NEXT: s_mov_b32 s49, s30
+; SI-NEXT: s_and_b32 s46, s30, 0xffff
+; SI-NEXT: s_mov_b32 s38, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[44:45], 16
-; SI-NEXT: s_lshl_b32 s54, s5, 16
+; SI-NEXT: s_mov_b32 s79, s48
; SI-NEXT: s_or_b32 s46, s46, s44
-; SI-NEXT: s_mov_b32 s31, s34
-; SI-NEXT: s_and_b32 s44, s52, 0xffff
-; SI-NEXT: s_mov_b32 s36, s35
+; SI-NEXT: s_mov_b32 s31, s38
+; SI-NEXT: s_mov_b32 s48, s34
+; SI-NEXT: s_and_b32 s44, s34, 0xffff
+; SI-NEXT: s_mov_b32 s38, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
+; SI-NEXT: s_mov_b32 s11, s77
+; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
; SI-NEXT: s_or_b32 s44, s44, s42
-; SI-NEXT: s_mov_b32 s35, s36
-; SI-NEXT: s_and_b32 s42, s53, 0xffff
+; SI-NEXT: s_mov_b32 s35, s38
+; SI-NEXT: s_mov_b32 s51, s36
+; SI-NEXT: s_and_b32 s42, s36, 0xffff
; SI-NEXT: s_mov_b32 s38, s37
-; SI-NEXT: s_lshr_b64 s[36:37], s[54:55], 16
+; SI-NEXT: s_lshr_b64 s[36:37], s[52:53], 16
+; SI-NEXT: s_mov_b32 s77, s39
; SI-NEXT: s_mov_b32 s5, s61
; SI-NEXT: s_mov_b32 s61, s59
; SI-NEXT: s_mov_b32 s59, s57
; SI-NEXT: s_mov_b32 s57, s47
; SI-NEXT: s_mov_b32 s47, s45
; SI-NEXT: s_mov_b32 s45, s43
-; SI-NEXT: s_or_b32 s42, s42, s54
-; SI-NEXT: s_mov_b32 s43, s55
+; SI-NEXT: s_or_b32 s42, s42, s52
+; SI-NEXT: s_mov_b32 s43, s53
; SI-NEXT: s_mov_b32 s37, s38
; SI-NEXT: s_mov_b64 s[38:39], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s51, s36
+; SI-NEXT: s_mov_b32 s48, s34
+; SI-NEXT: s_mov_b32 s49, s30
+; SI-NEXT: s_mov_b32 s72, vcc_lo
+; SI-NEXT: s_mov_b32 s62, s94
; SI-NEXT: s_mov_b64 s[38:39], -1
+; SI-NEXT: v_writelane_b32 v27, s4, 20
; SI-NEXT: ; implicit-def: $sgpr40
-; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: v_writelane_b32 v27, s5, 21
; SI-NEXT: ; implicit-def: $sgpr14
-; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: ; implicit-def: $sgpr50
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr10
@@ -41243,11 +41680,13 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_and_b64 s[38:39], s[38:39], exec
; SI-NEXT: s_cselect_b32 s38, 1, 0
; SI-NEXT: s_cmp_lg_u32 s38, 1
+; SI-NEXT: v_readlane_b32 s38, v27, 20
+; SI-NEXT: v_readlane_b32 s39, v27, 21
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s53, s53, 3
-; SI-NEXT: v_readlane_b32 s4, v27, 18
-; SI-NEXT: s_and_b32 s5, s53, 0xffff
+; SI-NEXT: s_add_i32 s4, s51, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v27, 19
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s42, s4, 0x30000
@@ -41257,9 +41696,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s37, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s43, s4, 0x30000
-; SI-NEXT: s_add_i32 s52, s52, 3
+; SI-NEXT: s_add_i32 s4, s48, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 11
-; SI-NEXT: s_and_b32 s5, s52, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -41269,9 +41708,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s45, s4, 0x30000
-; SI-NEXT: s_add_i32 s51, s51, 3
+; SI-NEXT: s_add_i32 s4, s49, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 10
-; SI-NEXT: s_and_b32 s5, s51, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s46, s4, 0x30000
@@ -41281,9 +41720,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s47, s4, 0x30000
-; SI-NEXT: s_add_i32 s50, s50, 3
+; SI-NEXT: s_add_i32 s4, s72, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 9
-; SI-NEXT: s_and_b32 s5, s50, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
@@ -41293,9 +41732,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_add_i32 s49, s49, 3
+; SI-NEXT: s_add_i32 s4, s62, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 8
-; SI-NEXT: s_and_b32 s5, s49, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
@@ -41305,9 +41744,10 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_add_i32 s48, s48, 3
+; SI-NEXT: v_readlane_b32 s4, v27, 18
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 7
-; SI-NEXT: s_and_b32 s5, s48, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
@@ -41380,7 +41820,6 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s40, s16, 0x30000
; SI-NEXT: s_and_b32 s16, s17, 0xffff
; SI-NEXT: s_lshl_b32 s17, s63, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
@@ -41391,11 +41830,10 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s11, s11, 0x30000
; SI-NEXT: s_add_i32 s12, s12, 0x30000
; SI-NEXT: s_add_i32 s13, s13, 0x30000
+; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s14, s14, 0x30000
; SI-NEXT: s_add_i32 s15, s15, 0x30000
; SI-NEXT: s_add_i32 s41, s16, 0x30000
-; SI-NEXT: s_lshr_b64 s[62:63], s[40:41], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[76:77], s[10:11], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[8:9], 16
@@ -41407,6 +41845,8 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 s[30:31], s[46:47], 16
; SI-NEXT: s_lshr_b64 s[34:35], s[44:45], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[42:43], 16
+; SI-NEXT: s_lshr_b64 s[38:39], s[40:41], 16
+; SI-NEXT: s_lshr_b64 s[50:51], s[14:15], 16
; SI-NEXT: s_lshr_b32 s63, s41, 16
; SI-NEXT: s_lshr_b32 s73, s15, 16
; SI-NEXT: s_lshr_b32 s75, s13, 16
@@ -41422,13 +41862,13 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s37, s43, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s16, s40, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s62, 16
+; SI-NEXT: s_lshl_b32 s17, s38, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s41, 0xffff
; SI-NEXT: s_lshl_b32 s18, s63, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s72, 16
+; SI-NEXT: s_lshl_b32 s18, s50, 16
; SI-NEXT: s_or_b32 s14, s14, s18
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_lshl_b32 s18, s73, 16
@@ -41499,7 +41939,7 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s29, s43, 0xffff
; SI-NEXT: s_lshl_b32 s40, s37, 16
; SI-NEXT: s_or_b32 s29, s29, s40
-; SI-NEXT: v_readlane_b32 s30, v26, 14
+; SI-NEXT: v_readlane_b32 s30, v26, 12
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s17
; SI-NEXT: v_mov_b32_e32 v2, s14
@@ -41526,9 +41966,7 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v23, s27
; SI-NEXT: v_mov_b32_e32 v24, s28
; SI-NEXT: v_mov_b32_e32 v25, s29
-; SI-NEXT: v_readlane_b32 s31, v26, 15
-; SI-NEXT: v_readlane_b32 s55, v26, 13
-; SI-NEXT: v_readlane_b32 s54, v26, 12
+; SI-NEXT: v_readlane_b32 s31, v26, 13
; SI-NEXT: v_readlane_b32 s53, v26, 11
; SI-NEXT: v_readlane_b32 s52, v26, 10
; SI-NEXT: v_readlane_b32 s51, v26, 9
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
index d259d3d97aca12..276916ef0f93ae 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
@@ -6596,8 +6596,9 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -6616,24 +6617,30 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -6641,27 +6648,20 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -6669,102 +6669,133 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB15_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB15_3
; SI-NEXT: .LBB15_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB15_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -6773,42 +6804,46 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -6841,9 +6876,9 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -6880,39 +6915,45 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB15_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -6941,17 +6982,7 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -6972,6 +7003,7 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -7001,170 +7033,184 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB15_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -7173,52 +7219,52 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -7230,25 +7276,29 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -7258,63 +7308,71 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB15_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -7343,19 +7401,7 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -10898,8 +10944,9 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -10918,153 +10965,181 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -11073,9 +11148,9 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -11085,14 +11160,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -11102,7 +11177,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -11111,11 +11186,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -11126,7 +11201,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -11135,11 +11210,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -11150,7 +11225,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -11159,7 +11234,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -11182,11 +11257,13 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -11195,10 +11272,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -11206,12 +11284,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -11219,10 +11299,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -11230,12 +11311,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -11243,10 +11326,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -11254,12 +11338,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -11267,10 +11353,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -11278,12 +11365,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -11332,18 +11421,8 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -11364,6 +11443,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -11393,174 +11473,188 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -11570,117 +11664,133 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -11719,20 +11829,8 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -18149,8 +18247,9 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -18169,24 +18268,30 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -18194,27 +18299,20 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -18222,102 +18320,133 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB31_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB31_3
; SI-NEXT: .LBB31_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB31_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -18326,42 +18455,46 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -18394,9 +18527,9 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -18433,39 +18566,45 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB31_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -18494,17 +18633,7 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -18525,6 +18654,7 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -18554,170 +18684,184 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB31_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB31_3
; VI-NEXT: .LBB31_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB31_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -18726,52 +18870,52 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -18783,25 +18927,29 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -18811,63 +18959,71 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB31_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -18896,19 +19052,7 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -22812,8 +22956,9 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -22832,153 +22977,181 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -22987,9 +23160,9 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -22999,14 +23172,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -23016,7 +23189,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -23025,11 +23198,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -23040,7 +23213,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -23049,11 +23222,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -23064,7 +23237,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -23073,7 +23246,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -23096,11 +23269,13 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -23109,10 +23284,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -23120,12 +23296,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -23133,10 +23311,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -23144,12 +23323,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -23157,10 +23338,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -23168,12 +23350,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -23181,10 +23365,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -23192,12 +23377,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -23246,18 +23433,8 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -23278,6 +23455,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -23307,174 +23485,188 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -23484,117 +23676,133 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -23633,20 +23841,8 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -28282,8 +28478,9 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -28302,24 +28499,30 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -28327,27 +28530,20 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -28355,102 +28551,133 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB43_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB43_3
; SI-NEXT: .LBB43_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB43_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -28459,42 +28686,46 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -28527,9 +28758,9 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -28566,39 +28797,45 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB43_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -28627,17 +28864,7 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -28658,6 +28885,7 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -28687,170 +28915,184 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB43_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB43_3
; VI-NEXT: .LBB43_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB43_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -28859,52 +29101,52 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -28916,25 +29158,29 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -28944,63 +29190,71 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB43_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -29029,19 +29283,7 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -32631,8 +32873,9 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -32651,153 +32894,181 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -32806,9 +33077,9 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -32818,14 +33089,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -32835,7 +33106,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -32844,11 +33115,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -32859,7 +33130,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -32868,11 +33139,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -32883,7 +33154,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -32892,7 +33163,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -32915,11 +33186,13 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -32928,10 +33201,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -32939,12 +33213,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -32952,10 +33228,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -32963,12 +33240,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -32976,10 +33255,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -32987,12 +33267,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -33000,10 +33282,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -33011,12 +33294,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -33065,18 +33350,8 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -33097,6 +33372,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -33126,174 +33402,188 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -33303,117 +33593,133 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -33452,20 +33758,8 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -37126,8 +37420,9 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -37146,24 +37441,30 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -37171,27 +37472,20 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -37199,102 +37493,133 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB51_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB51_3
; SI-NEXT: .LBB51_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB51_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -37303,42 +37628,46 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -37371,9 +37700,9 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -37410,39 +37739,45 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB51_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -37471,17 +37806,7 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -37502,6 +37827,7 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -37531,170 +37857,184 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB51_3
; VI-NEXT: .LBB51_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB51_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -37703,52 +38043,52 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -37760,25 +38100,29 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -37788,63 +38132,71 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB51_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -37873,19 +38225,7 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -41682,8 +42022,9 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -41702,153 +42043,181 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -41857,9 +42226,9 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -41869,14 +42238,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -41886,7 +42255,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -41895,11 +42264,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -41910,7 +42279,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -41919,11 +42288,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -41934,7 +42303,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -41943,7 +42312,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -41966,11 +42335,13 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -41979,10 +42350,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -41990,12 +42362,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -42003,10 +42377,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -42014,12 +42389,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -42027,10 +42404,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -42038,12 +42416,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -42051,10 +42431,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -42062,12 +42443,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -42116,18 +42499,8 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -42148,6 +42521,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -42177,174 +42551,188 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -42354,117 +42742,133 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -42503,20 +42907,8 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -44303,35 +44695,36 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v28, s64, 14
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
-; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s30, 24
-; SI-NEXT: v_writelane_b32 v28, s31, 25
-; SI-NEXT: v_readfirstlane_b32 s53, v0
-; SI-NEXT: v_readfirstlane_b32 s55, v2
-; SI-NEXT: s_lshr_b32 s45, s53, 16
+; SI-NEXT: v_writelane_b32 v28, s68, 17
+; SI-NEXT: v_writelane_b32 v28, s69, 18
+; SI-NEXT: v_writelane_b32 v28, s70, 19
+; SI-NEXT: v_writelane_b32 v28, s71, 20
+; SI-NEXT: v_writelane_b32 v28, s80, 21
+; SI-NEXT: v_writelane_b32 v28, s81, 22
+; SI-NEXT: v_writelane_b32 v28, s82, 23
+; SI-NEXT: v_writelane_b32 v28, s83, 24
+; SI-NEXT: v_writelane_b32 v28, s30, 25
+; SI-NEXT: v_writelane_b32 v28, s31, 26
+; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: s_lshr_b32 s45, s94, 16
; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s65, v4
-; SI-NEXT: s_lshr_b32 s44, s55, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v4
+; SI-NEXT: s_lshr_b32 s44, vcc_lo, 16
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v29, s45, 0
-; SI-NEXT: v_readfirstlane_b32 s66, v6
-; SI-NEXT: s_lshr_b32 s42, s65, 16
+; SI-NEXT: v_readfirstlane_b32 s34, v6
+; SI-NEXT: s_lshr_b32 s42, s30, 16
; SI-NEXT: v_writelane_b32 v29, s44, 1
-; SI-NEXT: v_readfirstlane_b32 s68, v8
-; SI-NEXT: s_lshr_b32 s40, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s36, v8
+; SI-NEXT: s_lshr_b32 s40, s34, 16
; SI-NEXT: v_writelane_b32 v29, s42, 2
-; SI-NEXT: s_lshr_b32 s14, s68, 16
+; SI-NEXT: s_lshr_b32 s14, s36, 16
; SI-NEXT: v_writelane_b32 v29, s40, 3
-; SI-NEXT: v_readfirstlane_b32 s70, v10
+; SI-NEXT: v_readfirstlane_b32 s38, v10
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v29, s14, 4
-; SI-NEXT: s_lshr_b32 s12, s70, 16
+; SI-NEXT: s_lshr_b32 s12, s38, 16
; SI-NEXT: v_writelane_b32 v29, s43, 5
; SI-NEXT: v_readfirstlane_b32 s41, v3
; SI-NEXT: v_writelane_b32 v29, s12, 6
@@ -44341,26 +44734,27 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v29, s15, 8
; SI-NEXT: v_readfirstlane_b32 s11, v9
; SI-NEXT: v_writelane_b32 v29, s13, 9
-; SI-NEXT: v_readfirstlane_b32 s7, v13
-; SI-NEXT: v_readfirstlane_b32 s80, v12
; SI-NEXT: v_readfirstlane_b32 s9, v11
; SI-NEXT: v_writelane_b32 v29, s11, 10
+; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: v_readfirstlane_b32 s50, v12
+; SI-NEXT: v_writelane_b32 v29, s9, 11
; SI-NEXT: s_lshr_b32 s93, s29, 16
-; SI-NEXT: s_lshr_b32 s81, s28, 16
+; SI-NEXT: s_lshr_b32 s66, s28, 16
; SI-NEXT: s_lshr_b32 s91, s27, 16
-; SI-NEXT: s_lshr_b32 s71, s26, 16
+; SI-NEXT: s_lshr_b32 s65, s26, 16
; SI-NEXT: s_lshr_b32 s89, s25, 16
-; SI-NEXT: s_lshr_b32 s69, s24, 16
+; SI-NEXT: s_lshr_b32 s64, s24, 16
; SI-NEXT: s_lshr_b32 s79, s23, 16
-; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s55, s22, 16
; SI-NEXT: s_lshr_b32 s77, s21, 16
-; SI-NEXT: s_lshr_b32 s64, s20, 16
+; SI-NEXT: s_lshr_b32 s54, s20, 16
; SI-NEXT: s_lshr_b32 s75, s19, 16
-; SI-NEXT: s_lshr_b32 s54, s18, 16
+; SI-NEXT: s_lshr_b32 s53, s18, 16
; SI-NEXT: s_lshr_b32 s73, s17, 16
; SI-NEXT: s_lshr_b32 s52, s16, 16
; SI-NEXT: s_lshr_b32 s49, s7, 16
-; SI-NEXT: s_lshr_b32 s5, s80, 16
+; SI-NEXT: s_lshr_b32 s5, s50, 16
; SI-NEXT: s_lshr_b32 s39, s9, 16
; SI-NEXT: s_lshr_b32 s37, s11, 16
; SI-NEXT: s_lshr_b32 s35, s13, 16
@@ -44368,10 +44762,10 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 vcc_hi, s41, 16
; SI-NEXT: s_lshr_b32 s95, s43, 16
; SI-NEXT: v_readfirstlane_b32 s4, v14
-; SI-NEXT: v_writelane_b32 v29, s9, 11
-; SI-NEXT: s_cmp_lg_u32 s4, 0
; SI-NEXT: v_writelane_b32 v29, s7, 12
-; SI-NEXT: v_writelane_b32 v29, s5, 13
+; SI-NEXT: s_cmp_lg_u32 s4, 0
+; SI-NEXT: v_writelane_b32 v29, s94, 13
+; SI-NEXT: v_writelane_b32 v29, s5, 14
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_lshl_b32 s46, s45, 16
@@ -44417,88 +44811,93 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s13, s37, 16
; SI-NEXT: s_or_b32 s10, s4, s72
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s74, s54, 16
+; SI-NEXT: s_lshl_b32 s74, s53, 16
; SI-NEXT: s_or_b32 s15, s11, s13
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s11, s39, 16
; SI-NEXT: s_or_b32 s8, s4, s74
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s76, s64, 16
+; SI-NEXT: s_lshl_b32 s76, s54, 16
; SI-NEXT: s_or_b32 s13, s9, s11
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s49, 16
-; SI-NEXT: s_or_b32 s6, s4, s76
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s62, s67, 16
-; SI-NEXT: s_mov_b32 s50, s49
-; SI-NEXT: s_or_b32 s49, s7, s9
; SI-NEXT: s_mov_b32 s11, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
-; SI-NEXT: s_mov_b32 s9, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
+; SI-NEXT: s_or_b32 s6, s4, s76
+; SI-NEXT: s_and_b32 s4, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s55, 16
+; SI-NEXT: v_writelane_b32 v29, s72, 15
; SI-NEXT: s_or_b32 s4, s4, s62
-; SI-NEXT: s_lshl_b32 s60, s69, 16
+; SI-NEXT: s_lshl_b32 s60, s64, 16
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s9, s49, 16
+; SI-NEXT: v_writelane_b32 v29, s73, 16
; SI-NEXT: s_mov_b32 s73, s78
+; SI-NEXT: s_mov_b32 s72, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[62:63], 16
+; SI-NEXT: s_and_b32 s62, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s65, 16
+; SI-NEXT: s_mov_b32 s51, s49
+; SI-NEXT: s_or_b32 s49, s7, s9
+; SI-NEXT: s_mov_b32 s9, s75
+; SI-NEXT: s_lshr_b64 s[80:81], s[74:75], 16
; SI-NEXT: s_mov_b32 s75, s88
; SI-NEXT: s_mov_b32 s7, s77
; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
-; SI-NEXT: s_mov_b32 s88, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[62:63], 16
-; SI-NEXT: s_and_b32 s62, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s58, s71, 16
-; SI-NEXT: s_mov_b32 s77, s90
-; SI-NEXT: s_mov_b32 s79, s88
+; SI-NEXT: s_mov_b32 s79, s72
; SI-NEXT: s_or_b32 s62, s62, s60
-; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s72, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[60:61], 16
; SI-NEXT: s_and_b32 s60, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s81, 16
-; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_lshl_b32 s56, s66, 16
+; SI-NEXT: s_mov_b32 s77, s90
+; SI-NEXT: s_mov_b32 s89, s72
; SI-NEXT: s_or_b32 s60, s60, s58
-; SI-NEXT: s_mov_b32 s92, s91
+; SI-NEXT: s_mov_b32 s72, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[58:59], 16
; SI-NEXT: s_and_b32 s58, s28, 0xffff
-; SI-NEXT: s_mov_b32 s91, s92
+; SI-NEXT: s_mov_b32 s91, s72
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 s94, s93
+; SI-NEXT: s_mov_b32 s72, s93
; SI-NEXT: s_lshr_b64 s[92:93], s[56:57], 16
-; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_mov_b32 s93, s94
+; SI-NEXT: s_mov_b32 s93, s72
; SI-NEXT: s_or_b32 s56, s56, s46
-; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s72, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[46:47], 16
-; SI-NEXT: s_and_b32 s46, s55, 0xffff
+; SI-NEXT: s_and_b32 s46, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 s95, s72
+; SI-NEXT: s_mov_b32 s72, vcc_lo
; SI-NEXT: s_or_b32 s46, s46, s44
-; SI-NEXT: s_mov_b32 s30, vcc_hi
+; SI-NEXT: s_mov_b32 s74, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[44:45], 16
-; SI-NEXT: s_and_b32 s44, s65, 0xffff
+; SI-NEXT: s_and_b32 s44, s30, 0xffff
; SI-NEXT: s_lshl_b32 s40, s40, 16
-; SI-NEXT: s_mov_b32 vcc_hi, s30
; SI-NEXT: s_or_b32 s44, s44, s42
-; SI-NEXT: s_mov_b32 s34, s31
-; SI-NEXT: s_lshr_b64 s[30:31], s[42:43], 16
-; SI-NEXT: s_and_b32 s42, s66, 0xffff
+; SI-NEXT: s_lshr_b64 s[82:83], s[42:43], 16
+; SI-NEXT: s_and_b32 s42, s34, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
-; SI-NEXT: s_mov_b32 s31, s34
+; SI-NEXT: s_mov_b32 vcc_hi, s74
+; SI-NEXT: s_mov_b32 s74, s30
+; SI-NEXT: s_mov_b32 s69, s34
; SI-NEXT: s_or_b32 s42, s42, s40
-; SI-NEXT: s_mov_b32 s36, s35
+; SI-NEXT: s_mov_b32 s30, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[40:41], 16
-; SI-NEXT: s_and_b32 s40, s68, 0xffff
+; SI-NEXT: s_and_b32 s40, s36, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_mov_b32 s35, s36
+; SI-NEXT: s_mov_b32 s35, s30
+; SI-NEXT: s_mov_b32 s68, s36
; SI-NEXT: s_or_b32 s40, s40, s14
-; SI-NEXT: s_mov_b32 s38, s37
+; SI-NEXT: s_mov_b32 s30, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[14:15], 16
-; SI-NEXT: s_and_b32 s14, s70, 0xffff
+; SI-NEXT: s_and_b32 s14, s38, 0xffff
; SI-NEXT: s_lshl_b32 s48, s5, 16
-; SI-NEXT: s_mov_b32 s37, s38
+; SI-NEXT: s_mov_b32 s37, s30
+; SI-NEXT: s_mov_b32 s71, s38
; SI-NEXT: s_or_b32 s14, s14, s12
-; SI-NEXT: s_mov_b32 s51, s39
+; SI-NEXT: s_mov_b32 s30, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[12:13], 16
-; SI-NEXT: s_and_b32 s12, s80, 0xffff
+; SI-NEXT: s_and_b32 s12, s50, 0xffff
; SI-NEXT: s_mov_b32 s5, s63
; SI-NEXT: s_mov_b32 s63, s61
; SI-NEXT: s_mov_b32 s61, s59
@@ -44512,16 +44911,25 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s12, s12, s48
; SI-NEXT: s_mov_b32 s13, s49
; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
-; SI-NEXT: s_mov_b32 s39, s51
-; SI-NEXT: s_mov_b32 s49, s50
+; SI-NEXT: s_mov_b32 s39, s30
+; SI-NEXT: s_mov_b32 s70, s50
+; SI-NEXT: s_mov_b32 s49, s51
; SI-NEXT: s_mov_b64 s[50:51], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s70, s50
+; SI-NEXT: s_mov_b32 s71, s38
+; SI-NEXT: s_mov_b32 s68, s36
+; SI-NEXT: s_mov_b32 s69, s34
+; SI-NEXT: s_mov_b32 s74, s30
+; SI-NEXT: s_mov_b32 s72, vcc_lo
; SI-NEXT: s_mov_b64 s[50:51], -1
+; SI-NEXT: v_writelane_b32 v29, s4, 15
; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: v_writelane_b32 v29, s5, 16
; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: ; implicit-def: $sgpr80
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $sgpr4
@@ -44537,7 +44945,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr46
; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr40
@@ -44548,13 +44956,15 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr48
; SI-NEXT: .LBB57_3: ; %Flow
; SI-NEXT: s_and_b64 s[50:51], s[50:51], exec
-; SI-NEXT: s_cselect_b32 s50, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s50, 1
+; SI-NEXT: s_cselect_b32 s30, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s30, 1
+; SI-NEXT: v_readlane_b32 s50, v29, 15
+; SI-NEXT: v_readlane_b32 s51, v29, 16
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s80, s80, 3
-; SI-NEXT: v_readlane_b32 s4, v29, 13
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
+; SI-NEXT: s_add_i32 s4, s70, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v29, 14
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s12, s4, 0x30000
@@ -44564,9 +44974,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s49, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s13, s4, 0x30000
-; SI-NEXT: s_add_i32 s70, s70, 3
+; SI-NEXT: s_add_i32 s4, s71, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 6
-; SI-NEXT: s_and_b32 s5, s70, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s14, s4, 0x30000
@@ -44576,9 +44986,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s39, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s15, s4, 0x30000
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s68, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 4
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
@@ -44588,9 +44998,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s37, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s41, s4, 0x30000
-; SI-NEXT: s_add_i32 s66, s66, 3
+; SI-NEXT: s_add_i32 s4, s69, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 3
-; SI-NEXT: s_and_b32 s5, s66, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s42, s4, 0x30000
@@ -44600,9 +45010,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s43, s4, 0x30000
-; SI-NEXT: s_add_i32 s65, s65, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 2
-; SI-NEXT: s_and_b32 s5, s65, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -44612,9 +45022,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s45, s4, 0x30000
-; SI-NEXT: s_add_i32 s55, s55, 3
+; SI-NEXT: s_add_i32 s4, s72, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 1
-; SI-NEXT: s_and_b32 s5, s55, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s46, s4, 0x30000
@@ -44624,9 +45034,10 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s47, s4, 0x30000
-; SI-NEXT: s_add_i32 s53, s53, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 0
-; SI-NEXT: s_and_b32 s5, s53, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
@@ -44638,7 +45049,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s66, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s58, s4, 0x30000
@@ -44648,7 +45059,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s59, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s65, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s60, s4, 0x30000
@@ -44658,7 +45069,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s61, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s62, s4, 0x30000
@@ -44668,7 +45079,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
+; SI-NEXT: s_lshl_b32 s5, s55, 16
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s23, 0xffff
@@ -44676,7 +45087,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_or_b32 s5, s6, s5
; SI-NEXT: s_and_b32 s6, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s64, 16
+; SI-NEXT: s_lshl_b32 s7, s54, 16
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_or_b32 s6, s7, s6
; SI-NEXT: s_and_b32 s7, s21, 0xffff
@@ -44684,7 +45095,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s7, s8, s7
; SI-NEXT: s_and_b32 s8, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s54, 16
+; SI-NEXT: s_lshl_b32 s9, s53, 16
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_or_b32 s8, s9, s8
; SI-NEXT: s_and_b32 s9, s19, 0xffff
@@ -44697,17 +45108,15 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s10, s11, s10
; SI-NEXT: s_and_b32 s11, s17, 0xffff
; SI-NEXT: s_lshl_b32 s16, s73, 16
-; SI-NEXT: s_or_b32 s11, s16, s11
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
; SI-NEXT: s_add_i32 s7, s7, 0x30000
+; SI-NEXT: s_or_b32 s11, s16, s11
; SI-NEXT: s_add_i32 s8, s8, 0x30000
; SI-NEXT: s_add_i32 s9, s9, 0x30000
; SI-NEXT: s_add_i32 s10, s10, 0x30000
; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_lshr_b64 s[72:73], s[10:11], 16
-; SI-NEXT: s_lshr_b64 s[74:75], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[76:77], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[4:5], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[62:63], 16
@@ -44715,11 +45124,13 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
; SI-NEXT: s_lshr_b64 s[94:95], s[56:57], 16
; SI-NEXT: s_lshr_b64 vcc, s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[30:31], s[44:45], 16
; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[38:39], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[12:13], 16
+; SI-NEXT: s_lshr_b64 s[50:51], s[10:11], 16
+; SI-NEXT: s_lshr_b64 s[80:81], s[8:9], 16
+; SI-NEXT: s_lshr_b64 s[82:83], s[44:45], 16
; SI-NEXT: s_lshr_b32 s73, s11, 16
; SI-NEXT: s_lshr_b32 s75, s9, 16
; SI-NEXT: s_lshr_b32 s77, s7, 16
@@ -44736,13 +45147,13 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s49, s13, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_lshl_b32 s16, s50, 16
; SI-NEXT: s_or_b32 s10, s10, s16
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s16, s73, 16
; SI-NEXT: s_or_b32 s11, s11, s16
; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s74, 16
+; SI-NEXT: s_lshl_b32 s16, s80, 16
; SI-NEXT: s_or_b32 s8, s8, s16
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s16, s75, 16
@@ -44790,7 +45201,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s26, vcc_hi, 16
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s26, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s27, s30, 16
+; SI-NEXT: s_lshl_b32 s27, s82, 16
; SI-NEXT: s_or_b32 s26, s26, s27
; SI-NEXT: s_and_b32 s27, s45, 0xffff
; SI-NEXT: s_lshl_b32 s28, s31, 16
@@ -44819,7 +45230,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s42, s49, 16
; SI-NEXT: s_or_b32 s13, s13, s42
-; SI-NEXT: v_readlane_b32 s30, v28, 24
+; SI-NEXT: v_readlane_b32 s30, v28, 25
; SI-NEXT: v_mov_b32_e32 v0, s10
; SI-NEXT: v_mov_b32_e32 v1, s11
; SI-NEXT: v_mov_b32_e32 v2, s8
@@ -44848,14 +45259,15 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v25, s15
; SI-NEXT: v_mov_b32_e32 v26, s12
; SI-NEXT: v_mov_b32_e32 v27, s13
-; SI-NEXT: v_readlane_b32 s31, v28, 25
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
-; SI-NEXT: v_readlane_b32 s67, v28, 17
+; SI-NEXT: v_readlane_b32 s31, v28, 26
+; SI-NEXT: v_readlane_b32 s83, v28, 24
+; SI-NEXT: v_readlane_b32 s82, v28, 23
+; SI-NEXT: v_readlane_b32 s81, v28, 22
+; SI-NEXT: v_readlane_b32 s80, v28, 21
+; SI-NEXT: v_readlane_b32 s71, v28, 20
+; SI-NEXT: v_readlane_b32 s70, v28, 19
+; SI-NEXT: v_readlane_b32 s69, v28, 18
+; SI-NEXT: v_readlane_b32 s68, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
; SI-NEXT: v_readlane_b32 s64, v28, 14
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
index 1be266983c3503..4ea3066833e341 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
@@ -7079,8 +7079,9 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -7099,169 +7100,195 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB15_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB15_3
; SI-NEXT: .LBB15_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB15_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -7270,47 +7297,47 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -7330,25 +7357,28 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -7362,64 +7392,75 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB15_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -7450,21 +7491,7 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -7485,6 +7512,7 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -7514,180 +7542,198 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB15_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -7696,52 +7742,52 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -7756,28 +7802,35 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -7788,70 +7841,75 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB15_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -7882,19 +7940,7 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -11751,8 +11797,9 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -11771,167 +11818,197 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -11940,9 +12017,9 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -11952,7 +12029,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -11962,20 +12039,20 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -11985,7 +12062,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -11994,11 +12071,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -12009,7 +12086,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -12018,11 +12095,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -12033,7 +12110,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -12044,29 +12121,34 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -12078,10 +12160,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -12089,12 +12172,14 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -12102,10 +12187,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -12113,12 +12199,14 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -12126,10 +12214,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -12137,12 +12226,14 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -12150,10 +12241,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -12161,13 +12253,16 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -12215,22 +12310,8 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -12251,6 +12332,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -12280,184 +12362,200 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -12467,15 +12565,15 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -12483,113 +12581,129 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -12628,20 +12742,8 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -19526,8 +19628,9 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -19546,169 +19649,195 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB31_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB31_3
; SI-NEXT: .LBB31_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB31_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -19717,47 +19846,47 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -19777,25 +19906,28 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -19809,64 +19941,75 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB31_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -19897,21 +20040,7 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -19932,6 +20061,7 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -19961,180 +20091,198 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB31_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB31_3
; VI-NEXT: .LBB31_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB31_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -20143,52 +20291,52 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -20203,28 +20351,35 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -20235,70 +20390,75 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB31_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -20329,19 +20489,7 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -24578,8 +24726,9 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -24598,167 +24747,197 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -24767,9 +24946,9 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -24779,7 +24958,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -24789,20 +24968,20 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -24812,7 +24991,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -24821,11 +25000,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -24836,7 +25015,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -24845,11 +25024,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -24860,7 +25039,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -24871,29 +25050,34 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -24905,10 +25089,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -24916,12 +25101,14 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -24929,10 +25116,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -24940,12 +25128,14 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -24953,10 +25143,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -24964,12 +25155,14 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -24977,10 +25170,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -24988,13 +25182,16 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -25042,22 +25239,8 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -25078,6 +25261,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -25107,184 +25291,200 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -25294,15 +25494,15 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -25310,113 +25510,129 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -25455,20 +25671,8 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -30484,8 +30688,9 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -30504,169 +30709,195 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB43_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB43_3
; SI-NEXT: .LBB43_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB43_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -30675,47 +30906,47 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -30735,25 +30966,28 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -30767,64 +31001,75 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB43_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -30855,21 +31100,7 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -30890,6 +31121,7 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -30919,180 +31151,198 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB43_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB43_3
; VI-NEXT: .LBB43_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB43_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -31101,52 +31351,52 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -31161,28 +31411,35 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -31193,70 +31450,75 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB43_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -31287,19 +31549,7 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -35212,8 +35462,9 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -35232,167 +35483,197 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -35401,9 +35682,9 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -35413,7 +35694,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -35423,20 +35704,20 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -35446,7 +35727,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -35455,11 +35736,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -35470,7 +35751,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -35479,11 +35760,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -35494,7 +35775,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -35505,29 +35786,34 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -35539,10 +35825,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -35550,12 +35837,14 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -35563,10 +35852,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -35574,12 +35864,14 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -35587,10 +35879,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -35598,12 +35891,14 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -35611,10 +35906,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -35622,13 +35918,16 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -35676,22 +35975,8 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -35712,6 +35997,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -35741,184 +36027,200 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -35928,15 +36230,15 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -35944,113 +36246,129 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -36089,20 +36407,8 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -40082,8 +40388,9 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -40102,169 +40409,195 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB51_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB51_3
; SI-NEXT: .LBB51_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB51_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -40273,47 +40606,47 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -40333,25 +40666,28 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -40365,64 +40701,75 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB51_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -40453,21 +40800,7 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -40488,6 +40821,7 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -40517,180 +40851,198 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB51_3
; VI-NEXT: .LBB51_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB51_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -40699,52 +41051,52 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -40759,28 +41111,35 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -40791,70 +41150,75 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB51_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -40885,19 +41249,7 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -45020,8 +45372,9 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -45040,167 +45393,197 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -45209,9 +45592,9 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -45221,7 +45604,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -45231,20 +45614,20 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -45254,7 +45637,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -45263,11 +45646,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -45278,7 +45661,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -45287,11 +45670,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -45302,7 +45685,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -45313,29 +45696,34 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -45347,10 +45735,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -45358,12 +45747,14 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -45371,10 +45762,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -45382,12 +45774,14 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -45395,10 +45789,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -45406,12 +45801,14 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -45419,10 +45816,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -45430,13 +45828,16 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -45484,22 +45885,8 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -45520,6 +45907,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -45549,184 +45937,200 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -45736,15 +46140,15 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -45752,113 +46156,129 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -45897,20 +46317,8 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -47789,39 +48197,39 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v30, s68, 18
; SI-NEXT: v_writelane_b32 v30, s69, 19
; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
+; SI-NEXT: v_writelane_b32 v30, s80, 21
+; SI-NEXT: v_writelane_b32 v30, s81, 22
+; SI-NEXT: v_writelane_b32 v30, s82, 23
+; SI-NEXT: v_writelane_b32 v30, s83, 24
+; SI-NEXT: v_writelane_b32 v30, s84, 25
+; SI-NEXT: v_writelane_b32 v30, s85, 26
+; SI-NEXT: v_writelane_b32 v30, s86, 27
+; SI-NEXT: v_writelane_b32 v30, s87, 28
; SI-NEXT: v_writelane_b32 v30, s30, 29
; SI-NEXT: v_writelane_b32 v30, s31, 30
-; SI-NEXT: v_readfirstlane_b32 s64, v0
-; SI-NEXT: v_readfirstlane_b32 s65, v2
-; SI-NEXT: s_lshr_b32 s42, s64, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v0
+; SI-NEXT: v_readfirstlane_b32 s30, v2
+; SI-NEXT: s_lshr_b32 s42, vcc_lo, 16
; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s67, v4
-; SI-NEXT: s_lshr_b32 s40, s65, 16
+; SI-NEXT: v_readfirstlane_b32 s34, v4
+; SI-NEXT: s_lshr_b32 s40, s30, 16
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v31, s42, 0
-; SI-NEXT: v_readfirstlane_b32 s69, v6
-; SI-NEXT: s_lshr_b32 s14, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s36, v6
+; SI-NEXT: s_lshr_b32 s14, s34, 16
; SI-NEXT: v_writelane_b32 v31, s40, 1
-; SI-NEXT: v_readfirstlane_b32 s71, v8
-; SI-NEXT: s_lshr_b32 s12, s69, 16
+; SI-NEXT: v_readfirstlane_b32 s38, v8
+; SI-NEXT: s_lshr_b32 s12, s36, 16
; SI-NEXT: v_writelane_b32 v31, s14, 2
-; SI-NEXT: s_lshr_b32 s10, s71, 16
+; SI-NEXT: s_lshr_b32 s10, s38, 16
; SI-NEXT: v_writelane_b32 v31, s12, 3
-; SI-NEXT: v_readfirstlane_b32 s81, v10
+; SI-NEXT: v_readfirstlane_b32 s48, v10
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v31, s10, 4
-; SI-NEXT: v_readfirstlane_b32 s83, v12
-; SI-NEXT: s_lshr_b32 s8, s81, 16
+; SI-NEXT: v_readfirstlane_b32 s50, v12
+; SI-NEXT: s_lshr_b32 s8, s48, 16
; SI-NEXT: v_writelane_b32 v31, s43, 5
-; SI-NEXT: s_lshr_b32 s6, s83, 16
+; SI-NEXT: s_lshr_b32 s6, s50, 16
; SI-NEXT: v_writelane_b32 v31, s8, 6
; SI-NEXT: v_readfirstlane_b32 s41, v3
; SI-NEXT: v_writelane_b32 v31, s6, 7
@@ -47833,26 +48241,28 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v31, s13, 10
; SI-NEXT: v_readfirstlane_b32 s9, v11
; SI-NEXT: v_writelane_b32 v31, s11, 11
-; SI-NEXT: v_readfirstlane_b32 s74, v15
-; SI-NEXT: v_readfirstlane_b32 s85, v14
; SI-NEXT: v_readfirstlane_b32 s7, v13
; SI-NEXT: v_writelane_b32 v31, s9, 12
+; SI-NEXT: v_writelane_b32 v31, s7, 13
+; SI-NEXT: v_readfirstlane_b32 s74, v15
+; SI-NEXT: v_readfirstlane_b32 s54, v14
+; SI-NEXT: v_writelane_b32 v31, vcc_lo, 14
; SI-NEXT: s_lshr_b32 s95, s29, 16
-; SI-NEXT: s_lshr_b32 s86, s28, 16
+; SI-NEXT: s_lshr_b32 s70, s28, 16
; SI-NEXT: s_lshr_b32 s93, s27, 16
-; SI-NEXT: s_lshr_b32 s84, s26, 16
+; SI-NEXT: s_lshr_b32 s69, s26, 16
; SI-NEXT: s_lshr_b32 s91, s25, 16
-; SI-NEXT: s_lshr_b32 s82, s24, 16
+; SI-NEXT: s_lshr_b32 s68, s24, 16
; SI-NEXT: s_lshr_b32 s89, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
; SI-NEXT: s_lshr_b32 s79, s21, 16
-; SI-NEXT: s_lshr_b32 s70, s20, 16
+; SI-NEXT: s_lshr_b32 s66, s20, 16
; SI-NEXT: s_lshr_b32 s77, s19, 16
-; SI-NEXT: s_lshr_b32 s68, s18, 16
+; SI-NEXT: s_lshr_b32 s65, s18, 16
; SI-NEXT: s_lshr_b32 s75, s17, 16
-; SI-NEXT: s_lshr_b32 s66, s16, 16
+; SI-NEXT: s_lshr_b32 s64, s16, 16
; SI-NEXT: s_lshr_b32 s53, s74, 16
-; SI-NEXT: s_lshr_b32 s5, s85, 16
+; SI-NEXT: s_lshr_b32 s5, s54, 16
; SI-NEXT: s_lshr_b32 s51, s7, 16
; SI-NEXT: s_lshr_b32 s49, s9, 16
; SI-NEXT: s_lshr_b32 s39, s11, 16
@@ -47861,10 +48271,10 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s31, s41, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s43, 16
; SI-NEXT: v_readfirstlane_b32 s4, v16
-; SI-NEXT: v_writelane_b32 v31, s7, 13
+; SI-NEXT: v_writelane_b32 v31, s74, 15
; SI-NEXT: s_cmp_lg_u32 s4, 0
-; SI-NEXT: v_writelane_b32 v31, s74, 14
-; SI-NEXT: v_writelane_b32 v31, s5, 15
+; SI-NEXT: v_writelane_b32 v31, s5, 16
+; SI-NEXT: v_writelane_b32 v31, s30, 17
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s45, s17, 0xffff
@@ -47896,109 +48306,110 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s41, s41, s76
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_lshl_b32 s76, s35, 16
+; SI-NEXT: s_and_b32 s4, s16, 0xffff
+; SI-NEXT: s_lshl_b32 s72, s64, 16
; SI-NEXT: s_or_b32 s15, s15, s76
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s76, s37, 16
+; SI-NEXT: s_or_b32 s4, s4, s72
; SI-NEXT: s_or_b32 s13, s13, s76
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s76, s39, 16
+; SI-NEXT: s_lshl_b32 s52, s5, 16
+; SI-NEXT: s_mov_b32 s5, s73
+; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
; SI-NEXT: s_or_b32 s11, s11, s76
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s76, s49, 16
+; SI-NEXT: v_writelane_b32 v31, s72, 18
+; SI-NEXT: s_lshl_b32 s62, s65, 16
; SI-NEXT: s_or_b32 s9, s9, s76
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_lshl_b32 s76, s51, 16
-; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s72, s66, 16
+; SI-NEXT: v_writelane_b32 v31, s73, 19
+; SI-NEXT: s_and_b32 s72, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s66, 16
; SI-NEXT: s_or_b32 s7, s7, s76
; SI-NEXT: s_and_b32 s74, s74, 0xffff
; SI-NEXT: s_lshl_b32 s76, s53, 16
-; SI-NEXT: s_or_b32 s4, s4, s72
-; SI-NEXT: s_lshl_b32 s62, s68, 16
-; SI-NEXT: s_mov_b32 s54, s53
-; SI-NEXT: s_or_b32 s53, s74, s76
-; SI-NEXT: s_mov_b32 s76, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[72:73], 16
-; SI-NEXT: s_and_b32 s72, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s60, s70, 16
-; SI-NEXT: s_mov_b32 s75, s76
; SI-NEXT: s_or_b32 s72, s72, s62
-; SI-NEXT: s_mov_b32 s78, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[62:63], 16
+; SI-NEXT: s_lshr_b64 s[84:85], s[62:63], 16
; SI-NEXT: s_and_b32 s62, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s58, s80, 16
-; SI-NEXT: s_mov_b32 s77, s78
+; SI-NEXT: s_lshl_b32 s58, s67, 16
+; SI-NEXT: s_mov_b32 s55, s53
+; SI-NEXT: s_or_b32 s53, s74, s76
; SI-NEXT: s_or_b32 s62, s62, s60
-; SI-NEXT: s_mov_b32 s88, s79
+; SI-NEXT: s_mov_b32 s74, s79
; SI-NEXT: s_lshr_b64 s[78:79], s[60:61], 16
; SI-NEXT: s_and_b32 s60, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s82, 16
-; SI-NEXT: s_mov_b32 s79, s88
+; SI-NEXT: s_lshl_b32 s56, s68, 16
+; SI-NEXT: s_mov_b32 s79, s74
; SI-NEXT: s_or_b32 s60, s60, s58
-; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s74, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[58:59], 16
; SI-NEXT: s_and_b32 s58, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s46, s84, 16
-; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_lshl_b32 s46, s69, 16
+; SI-NEXT: s_mov_b32 s89, s74
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 s92, s91
+; SI-NEXT: s_mov_b32 s74, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[56:57], 16
; SI-NEXT: s_and_b32 s56, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s86, 16
-; SI-NEXT: s_mov_b32 s91, s92
+; SI-NEXT: s_lshl_b32 s44, s70, 16
+; SI-NEXT: s_mov_b32 s91, s74
; SI-NEXT: s_or_b32 s56, s56, s46
-; SI-NEXT: s_mov_b32 s94, s93
+; SI-NEXT: s_mov_b32 s74, s93
; SI-NEXT: s_lshr_b64 s[92:93], s[46:47], 16
; SI-NEXT: s_and_b32 s46, s28, 0xffff
; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_mov_b32 s93, s94
+; SI-NEXT: s_mov_b32 s93, s74
; SI-NEXT: s_or_b32 s46, s46, s44
-; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s74, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[44:45], 16
-; SI-NEXT: s_and_b32 s44, s64, 0xffff
+; SI-NEXT: s_and_b32 s44, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s40, s40, 16
-; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 s95, s74
; SI-NEXT: s_or_b32 s44, s44, s42
-; SI-NEXT: s_mov_b32 s30, vcc_hi
+; SI-NEXT: s_mov_b32 s74, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[42:43], 16
-; SI-NEXT: s_and_b32 s42, s65, 0xffff
+; SI-NEXT: s_and_b32 s42, s30, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
-; SI-NEXT: s_mov_b32 vcc_hi, s30
+; SI-NEXT: s_mov_b32 vcc_hi, s74
; SI-NEXT: s_or_b32 s42, s42, s40
-; SI-NEXT: s_mov_b32 s34, s31
+; SI-NEXT: s_mov_b32 s74, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[40:41], 16
-; SI-NEXT: s_and_b32 s40, s67, 0xffff
+; SI-NEXT: s_and_b32 s40, s34, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_mov_b32 s31, s34
+; SI-NEXT: s_mov_b32 s31, s74
+; SI-NEXT: s_mov_b32 s74, s34
; SI-NEXT: s_or_b32 s40, s40, s14
-; SI-NEXT: s_mov_b32 s36, s35
+; SI-NEXT: s_mov_b32 s76, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[14:15], 16
-; SI-NEXT: s_and_b32 s14, s69, 0xffff
+; SI-NEXT: s_and_b32 s14, s36, 0xffff
; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_mov_b32 s35, s36
; SI-NEXT: s_or_b32 s14, s14, s12
-; SI-NEXT: s_mov_b32 s38, s37
-; SI-NEXT: s_lshr_b64 s[36:37], s[12:13], 16
-; SI-NEXT: s_and_b32 s12, s71, 0xffff
+; SI-NEXT: s_lshr_b64 s[86:87], s[12:13], 16
+; SI-NEXT: s_and_b32 s12, s38, 0xffff
; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_mov_b32 s37, s38
+; SI-NEXT: s_mov_b32 s35, s76
+; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: s_mov_b32 s81, s38
; SI-NEXT: s_or_b32 s12, s12, s10
-; SI-NEXT: s_mov_b32 s48, s39
+; SI-NEXT: s_mov_b32 s36, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[10:11], 16
-; SI-NEXT: s_and_b32 s10, s81, 0xffff
+; SI-NEXT: s_and_b32 s10, s48, 0xffff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_mov_b32 s39, s48
+; SI-NEXT: s_mov_b32 s39, s36
+; SI-NEXT: s_mov_b32 s80, s48
; SI-NEXT: s_or_b32 s10, s10, s8
-; SI-NEXT: s_mov_b32 s50, s49
+; SI-NEXT: s_mov_b32 s36, s49
; SI-NEXT: s_lshr_b64 s[48:49], s[8:9], 16
-; SI-NEXT: s_and_b32 s8, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s52, s5, 16
-; SI-NEXT: s_mov_b32 s49, s50
+; SI-NEXT: s_and_b32 s8, s50, 0xffff
+; SI-NEXT: s_mov_b32 s49, s36
+; SI-NEXT: s_mov_b32 s83, s50
; SI-NEXT: s_or_b32 s8, s8, s6
-; SI-NEXT: s_mov_b32 s55, s51
+; SI-NEXT: s_mov_b32 s36, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[6:7], 16
-; SI-NEXT: s_and_b32 s6, s85, 0xffff
-; SI-NEXT: s_mov_b32 s5, s73
+; SI-NEXT: s_and_b32 s6, s54, 0xffff
; SI-NEXT: s_mov_b32 s73, s63
; SI-NEXT: s_mov_b32 s63, s61
; SI-NEXT: s_mov_b32 s61, s59
@@ -48015,16 +48426,25 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s6, s6, s52
; SI-NEXT: s_mov_b32 s7, s53
; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
-; SI-NEXT: s_mov_b32 s51, s55
-; SI-NEXT: s_mov_b32 s53, s54
+; SI-NEXT: s_mov_b32 s51, s36
+; SI-NEXT: s_mov_b32 s82, s54
+; SI-NEXT: s_mov_b32 s53, s55
; SI-NEXT: s_mov_b64 s[54:55], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: s_mov_b32 s82, s54
+; SI-NEXT: s_mov_b32 s83, s50
+; SI-NEXT: s_mov_b32 s80, s48
+; SI-NEXT: s_mov_b32 s81, s38
+; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: s_mov_b32 s74, s34
; SI-NEXT: s_mov_b64 s[54:55], -1
+; SI-NEXT: v_writelane_b32 v31, s6, 18
; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: v_writelane_b32 v31, s7, 19
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr62
; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr60
@@ -48042,7 +48462,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr14
-; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr86
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr38
; SI-NEXT: ; implicit-def: $sgpr10
@@ -48053,25 +48473,27 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr52
; SI-NEXT: .LBB57_3: ; %Flow
; SI-NEXT: s_and_b64 s[54:55], s[54:55], exec
-; SI-NEXT: s_cselect_b32 s54, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s54, 1
+; SI-NEXT: s_cselect_b32 s36, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s36, 1
+; SI-NEXT: v_readlane_b32 s54, v31, 18
+; SI-NEXT: v_readlane_b32 s55, v31, 19
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s85, s85, 3
-; SI-NEXT: v_readlane_b32 s4, v31, 15
-; SI-NEXT: s_and_b32 s5, s85, 0xffff
+; SI-NEXT: s_add_i32 s4, s82, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 16
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s6, s4, 0x30000
-; SI-NEXT: v_readlane_b32 s4, v31, 14
+; SI-NEXT: v_readlane_b32 s4, v31, 15
; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s53, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s7, s4, 0x30000
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s83, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 7
-; SI-NEXT: s_and_b32 s5, s83, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s8, s4, 0x30000
@@ -48081,9 +48503,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s51, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s9, s4, 0x30000
-; SI-NEXT: s_add_i32 s81, s81, 3
+; SI-NEXT: s_add_i32 s4, s80, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 6
-; SI-NEXT: s_and_b32 s5, s81, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s10, s4, 0x30000
@@ -48093,9 +48515,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s49, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s11, s4, 0x30000
-; SI-NEXT: s_add_i32 s71, s71, 3
+; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 4
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s12, s4, 0x30000
@@ -48105,9 +48527,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s39, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s13, s4, 0x30000
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: s_add_i32 s4, s76, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 3
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s14, s4, 0x30000
@@ -48117,9 +48539,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s37, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s15, s4, 0x30000
-; SI-NEXT: s_add_i32 s67, s67, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 2
-; SI-NEXT: s_and_b32 s5, s67, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
@@ -48129,9 +48551,10 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s41, s4, 0x30000
-; SI-NEXT: s_add_i32 s65, s65, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 1
-; SI-NEXT: s_and_b32 s5, s65, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s42, s4, 0x30000
@@ -48141,9 +48564,10 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s43, s4, 0x30000
-; SI-NEXT: s_add_i32 s64, s64, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 14
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 0
-; SI-NEXT: s_and_b32 s5, s64, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -48155,7 +48579,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s70, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
@@ -48165,7 +48589,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s69, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s56, s4, 0x30000
@@ -48175,7 +48599,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s68, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s58, s4, 0x30000
@@ -48185,7 +48609,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s59, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s60, s4, 0x30000
@@ -48195,7 +48619,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s61, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s66, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s62, s4, 0x30000
@@ -48205,7 +48629,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s65, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s72, s4, 0x30000
@@ -48215,7 +48639,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_add_i32 s73, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
@@ -48223,8 +48647,6 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b64 s[76:77], s[72:73], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[62:63], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[60:61], 16
; SI-NEXT: s_lshr_b64 s[90:91], s[58:59], 16
@@ -48233,11 +48655,13 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 vcc, s[44:45], 16
; SI-NEXT: s_lshr_b64 s[30:31], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[34:35], s[40:41], 16
-; SI-NEXT: s_lshr_b64 s[36:37], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[38:39], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[10:11], 16
; SI-NEXT: s_lshr_b64 s[50:51], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[52:53], s[6:7], 16
+; SI-NEXT: s_lshr_b64 s[54:55], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[84:85], s[72:73], 16
+; SI-NEXT: s_lshr_b64 s[86:87], s[14:15], 16
; SI-NEXT: s_lshr_b32 s75, s5, 16
; SI-NEXT: s_lshr_b32 s77, s73, 16
; SI-NEXT: s_lshr_b32 s79, s63, 16
@@ -48255,13 +48679,13 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s53, s7, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s74, 16
+; SI-NEXT: s_lshl_b32 s16, s54, 16
; SI-NEXT: s_or_b32 s4, s4, s16
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_lshl_b32 s16, s75, 16
; SI-NEXT: s_or_b32 s5, s5, s16
; SI-NEXT: s_and_b32 s16, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s76, 16
+; SI-NEXT: s_lshl_b32 s17, s84, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s73, 0xffff
; SI-NEXT: s_lshl_b32 s18, s77, 16
@@ -48315,7 +48739,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s44, s35, 16
; SI-NEXT: s_or_b32 s41, s41, s44
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s36, 16
+; SI-NEXT: s_lshl_b32 s44, s86, 16
; SI-NEXT: s_or_b32 s14, s14, s44
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_lshl_b32 s44, s37, 16
@@ -48376,14 +48800,14 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v28, s6
; SI-NEXT: v_mov_b32_e32 v29, s7
; SI-NEXT: v_readlane_b32 s31, v30, 30
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
+; SI-NEXT: v_readlane_b32 s87, v30, 28
+; SI-NEXT: v_readlane_b32 s86, v30, 27
+; SI-NEXT: v_readlane_b32 s85, v30, 26
+; SI-NEXT: v_readlane_b32 s84, v30, 25
+; SI-NEXT: v_readlane_b32 s83, v30, 24
+; SI-NEXT: v_readlane_b32 s82, v30, 23
+; SI-NEXT: v_readlane_b32 s81, v30, 22
+; SI-NEXT: v_readlane_b32 s80, v30, 21
; SI-NEXT: v_readlane_b32 s70, v30, 20
; SI-NEXT: v_readlane_b32 s69, v30, 19
; SI-NEXT: v_readlane_b32 s68, v30, 18
More information about the llvm-commits
mailing list