[llvm-branch-commits] [llvm] [AMDGPU] Retire dying uses before adding defs in GCNDownwardRPTracker (PR #228294)
Dhruva Chakrabarti via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Thu Oct 1 18:05:48 PDT 2026
https://github.com/dhruvachak created https://github.com/llvm/llvm-project/pull/228294
GCNDownwardRPTracker::advanceToNext() added an instruction's defs to the live set while the uses that die at that instruction were still in it, and only dropped them in the following advanceBeforeNext(). This could lead to inflated register pressure reporting.
Apply the transfer function in the order the generic tracker uses in RegPressureTracker::advance(): retire the lanes that die at the instruction first, then add its defs. Early-clobber defs are the exception. PHIs are skipped because nothing dies there. All of these are now done by advanceToNext().
advanceBeforeNext() now only drops dead def lanes, the uses having been retired already.
Two callers depend on the previous ordering and are updated:
- SIFormMemoryClauses::checkPressure() deliberately keeps dying uses live, because every def in a clause is early-clobber and stays live to the end of the clause. It now asks for that explicitly via RetireDeadUses=false.
- The next-use distance printer reports distances for every register live at the instruction, including the ones it consumes, so it adds the live-in lanes of the uses back.
This patch affects default codegen, not just the GCNTrackers path. This is because region pressure is measured with the downward tracker regardless of GCNTrackers.
The trackers become measurably more accurate: in regpressure_printer.mir the upward and downward trackers now report identical pressure for all ten functions, where five previously disagreed, and in schedule-regpressure-ilp-metric-spills.mir the GCN trackers stop emitting twelve spills and agree with the generic tracker that none are needed.
Among the LIT tests changed, 35 of 47 assembly tests have the same spill and instruction counts, differing only in numbering or order. Among the remaining, 5 have reduced instruction count. load-global-i16.ll is the most notable, where 32 of the 52 removed instructions are spill/reload pairs. rewrite-vgpr-mfma-to-agpr-spill-multi-store-codegen.ll no longer has regalloc spills that the rewrite phase eliminated. The number of AGPRs drops from 128 to 114 for this test. 6 tests grow by a handful. The only outlier is amdgcn.bitcast.1024bit.ll, at +115 combined spills and reloads. That is not a scheduling regression: for the worst function the post-scheduler MIR is identical in both builds, and the allocator diverges only because LiveIntervals is maintained incrementally. Recomputing live ranges would help but that is not part of this patch.
>From 560b66cc9e044041a22fa1f80daa07d8b8d9bb82 Mon Sep 17 00:00:00 2001
From: Dhruva Chakrabarti <Dhruva.Chakrabarti at amd.com>
Date: Fri, 25 Sep 2026 21:09:09 -0500
Subject: [PATCH] [AMDGPU] Retire dying uses before adding defs in
GCNDownwardRPTracker
GCNDownwardRPTracker::advanceToNext() added an instruction's defs to the
live set while the uses that die at that instruction were still in it,
and only dropped them in the following advanceBeforeNext(). This could
lead to inflated register pressure reporting.
Apply the transfer function in the order the generic tracker uses in
RegPressureTracker::advance(): retire the lanes that die at the
instruction first, then add its defs. Early-clobber defs are the
exception. PHIs are skipped because nothing dies there. All of these are
now done by advanceToNext().
advanceBeforeNext() now only drops dead def lanes, the uses having been
retired already.
Two callers depend on the previous ordering and are updated:
- SIFormMemoryClauses::checkPressure() deliberately keeps dying uses live,
because every def in a clause is early-clobber and stays live to the end
of the clause. It now asks for that explicitly via RetireDeadUses=false.
- The next-use distance printer reports distances for every register live
at the instruction, including the ones it consumes, so it adds the
live-in lanes of the uses back.
This patch affects default codegen, not just the GCNTrackers path. This is
because region pressure is measured with the downward tracker regardless
of GCNTrackers.
The trackers become measurably more accurate: in regpressure_printer.mir
the upward and downward trackers now report identical pressure for all
ten functions, where five previously disagreed, and in
schedule-regpressure-ilp-metric-spills.mir the GCN trackers stop emitting
twelve spills and agree with the generic tracker that none are needed.
Among the LIT tests changed, 35 of 47 assembly tests have the same spill
and instruction counts, differing only in numbering or order. Among the
remaining, 5 have reduced instruction count. load-global-i16.ll is the most
notable, where 32 of the 52 removed instructions are spill/reload pairs.
rewrite-vgpr-mfma-to-agpr-spill-multi-store-codegen.ll no longer has regalloc
spills that the rewrite phase eliminated. The number of AGPRs
drops from 128 to 114 for this test. 6 tests grow by a handful. The only
outlier is amdgcn.bitcast.1024bit.ll, at +115 combined spills and reloads.
That is not a scheduling regression: for the worst function the post-scheduler
MIR is identical in both builds, and the allocator diverges only because
LiveIntervals is maintained incrementally. Recomputing live ranges would help
but that is not part of this patch.
Co-authored-by: Cursor <cursoragent at cursor.com>
---
.../Target/AMDGPU/AMDGPUNextUseAnalysis.cpp | 19 +-
llvm/lib/Target/AMDGPU/GCNRegPressure.cpp | 57 +-
llvm/lib/Target/AMDGPU/GCNRegPressure.h | 5 +-
.../lib/Target/AMDGPU/SIFormMemoryClauses.cpp | 9 +-
llvm/test/CodeGen/AMDGPU/GlobalISel/fshl.ll | 316 +-
llvm/test/CodeGen/AMDGPU/GlobalISel/fshr.ll | 288 +-
.../test/CodeGen/AMDGPU/GlobalISel/uaddsat.ll | 4 +-
llvm/test/CodeGen/AMDGPU/addrspacecast.ll | 420 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll | 17306 ++++++++--------
.../CodeGen/AMDGPU/amdgcn.bitcast.256bit.ll | 504 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.384bit.ll | 136 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.448bit.ll | 152 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll | 6017 +++---
.../CodeGen/AMDGPU/amdgcn.bitcast.576bit.ll | 354 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll | 924 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll | 751 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll | 330 +-
llvm/test/CodeGen/AMDGPU/bf16.ll | 294 +-
.../callee-special-input-vgprs-packed.ll | 10 +-
.../AMDGPU/callee-special-input-vgprs.ll | 10 +-
.../AMDGPU/debug-value-scheduler-crash.mir | 38 +-
llvm/test/CodeGen/AMDGPU/div_i128.ll | 150 +-
llvm/test/CodeGen/AMDGPU/div_v2i128.ll | 462 +-
llvm/test/CodeGen/AMDGPU/fceil64.ll | 210 +-
llvm/test/CodeGen/AMDGPU/fmaximum.ll | 157 +-
llvm/test/CodeGen/AMDGPU/fminimum.ll | 157 +-
llvm/test/CodeGen/AMDGPU/freeze.ll | 769 +-
.../AMDGPU/gfx-callable-argument-types.ll | 6 +-
.../CodeGen/AMDGPU/high-RP-reschedule.mir | 6 +-
llvm/test/CodeGen/AMDGPU/llvm.exp.f64.ll | 570 +-
llvm/test/CodeGen/AMDGPU/llvm.exp10.f64.ll | 898 +-
llvm/test/CodeGen/AMDGPU/llvm.exp2.f64.ll | 702 +-
llvm/test/CodeGen/AMDGPU/llvm.maximum.f16.ll | 150 +-
llvm/test/CodeGen/AMDGPU/llvm.maximum.f32.ll | 216 +-
llvm/test/CodeGen/AMDGPU/llvm.maximum.f64.ll | 156 +-
llvm/test/CodeGen/AMDGPU/llvm.minimum.f16.ll | 150 +-
llvm/test/CodeGen/AMDGPU/llvm.minimum.f32.ll | 216 +-
llvm/test/CodeGen/AMDGPU/llvm.minimum.f64.ll | 156 +-
llvm/test/CodeGen/AMDGPU/load-constant-i32.ll | 80 +-
llvm/test/CodeGen/AMDGPU/load-global-i16.ll | 1240 +-
llvm/test/CodeGen/AMDGPU/load-global-i32.ll | 472 +-
llvm/test/CodeGen/AMDGPU/load-global-i8.ll | 464 +-
...ne-scheduler-sink-trivial-remats-debug.mir | 10 +-
.../machine-scheduler-sink-trivial-remats.mir | 104 +-
.../AMDGPU/promote-constOffset-to-imm.ll | 164 +-
.../CodeGen/AMDGPU/regpressure_printer.mir | 615 +-
llvm/test/CodeGen/AMDGPU/rem_i128.ll | 166 +-
...-mfma-to-agpr-spill-multi-store-codegen.ll | 824 +-
...ssert-dead-def-subreg-use-other-subreg.mir | 2 +-
.../AMDGPU/sched-dynamic-vgpr-blocks.mir | 6 +-
.../AMDGPU/sched_mfma_rewrite_copies.mir | 30 +-
...schedule-regpressure-ilp-metric-spills.mir | 9 +-
llvm/test/CodeGen/AMDGPU/spill-agpr.ll | 2 +-
llvm/test/CodeGen/AMDGPU/uaddsat.ll | 48 +-
.../test/CodeGen/AMDGPU/vector-reduce-fmax.ll | 84 +-
.../CodeGen/AMDGPU/vector-reduce-fmaximum.ll | 220 +-
.../test/CodeGen/AMDGPU/vector-reduce-fmin.ll | 84 +-
.../CodeGen/AMDGPU/vector-reduce-fminimum.ll | 220 +-
llvm/test/CodeGen/AMDGPU/vector-reduce-mul.ll | 276 +-
.../AMDGPU/vgpr-mark-last-scratch-load.ll | 8 +-
60 files changed, 19138 insertions(+), 19065 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUNextUseAnalysis.cpp b/llvm/lib/Target/AMDGPU/AMDGPUNextUseAnalysis.cpp
index a667b96c0d089..d880bfe6912bb 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUNextUseAnalysis.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUNextUseAnalysis.cpp
@@ -2451,11 +2451,26 @@ void printNextUseDistancesAsJson(json::OStream &J, const MachineFunction &MF,
RPTracker.reset(MI, MBB.end());
RPTracker.advance();
+ // Report distance for every register live at MI including the ones
+ // that the tracker retired. Add their live-in lanes back.
+ GCNRPTracker::LiveRegSet LiveAtMI = RPTracker.getLiveRegs();
+ if (!MI.isDebugInstr()) {
+ SlotIndex BaseIdx = LIS.getInstructionIndex(MI).getBaseIndex();
+ for (const MachineOperand &MO : MI.all_uses()) {
+ Register Reg = MO.getReg();
+ if (!Reg.isVirtual() || !MO.readsReg())
+ continue;
+ LaneBitmask Mask = getLiveLaneMask(Reg, BaseIdx, LIS, MRI);
+ if (Mask.any())
+ LiveAtMI[Reg] |= Mask;
+ }
+ }
+
UseDistancePair Furthest;
UseDistancePair FurthestSubreg;
RelevantUses.clear();
- NUA.getNextUseDistances(RPTracker.getLiveRegs(), MI, Furthest,
- &FurthestSubreg, &RelevantUses);
+ NUA.getNextUseDistances(LiveAtMI, MI, Furthest, &FurthestSubreg,
+ &RelevantUses);
J.objectBegin();
printInstrMember(J, MST, MI, NUAImpl);
diff --git a/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp b/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp
index a371c32af0786..b364a741cc012 100644
--- a/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp
@@ -720,18 +720,22 @@ bool GCNDownwardRPTracker::advanceBeforeNext(MachineInstr *MI,
assert(SI.isValid());
- // Remove dead registers or mask bits.
SmallSet<Register, 8> SeenRegs;
- for (auto &MO : CurrMI->operands()) {
- if (!MO.isReg() || !MO.getReg().isVirtual())
- continue;
- if (MO.isUse() && !MO.readsReg())
- continue;
- if (!UseInternalIterator && MO.isDef())
- continue;
- if (!SeenRegs.insert(MO.getReg()).second)
- continue;
- retireVirtReg(MO.getReg(), SI);
+ if (UseInternalIterator) {
+ // Drop dead def lanes.
+ for (const MachineOperand &MO : CurrMI->all_defs()) {
+ Register Reg = MO.getReg();
+ if (Reg.isVirtual() && SeenRegs.insert(Reg).second)
+ retireVirtReg(Reg, SI);
+ }
+ } else {
+ // Defs have not been added yet, so the only lanes that can be dropped are
+ // uses that are already dead.
+ for (const MachineOperand &MO : CurrMI->all_uses()) {
+ Register Reg = MO.getReg();
+ if (MO.readsReg() && Reg.isVirtual() && SeenRegs.insert(Reg).second)
+ retireVirtReg(Reg, SI);
+ }
}
MaxPressure = max(MaxPressure, CurPressure);
@@ -742,7 +746,8 @@ bool GCNDownwardRPTracker::advanceBeforeNext(MachineInstr *MI,
}
void GCNDownwardRPTracker::advanceToNext(MachineInstr *MI,
- bool UseInternalIterator) {
+ bool UseInternalIterator,
+ bool RetireDeadUses) {
if (UseInternalIterator) {
LastTrackedMI = &*NextMI++;
NextMI = skipDebugInstructionsForward(NextMI, MBBEnd);
@@ -752,6 +757,34 @@ void GCNDownwardRPTracker::advanceToNext(MachineInstr *MI,
const MachineInstr *CurrMI = LastTrackedMI;
+ if (RetireDeadUses) {
+ // Mirror ECDefPressure handling from GCNUpwardRPTracker::recede().
+ GCNRegPressure ECDefPressure;
+ for (const MachineOperand &MO : CurrMI->all_defs()) {
+ Register Reg = MO.getReg();
+ if (Reg.isVirtual() && MO.isEarlyClobber())
+ ECDefPressure.inc(Reg, LaneBitmask::getNone(), getDefRegMask(MO, *MRI),
+ *MRI);
+ }
+
+ MaxPressure = max(MaxPressure, CurPressure + ECDefPressure);
+
+ // Retire the uses that die here before adding the defs, mirroring
+ // RegPressureTracker::advance().
+ //
+ // A PHI is skipped since its operands do not die here.
+ if (!CurrMI->isPHI()) {
+ SlotIndex SI = LIS.getInstructionIndex(*CurrMI).getDeadSlot();
+ assert(SI.isValid());
+ SmallSet<Register, 8> SeenRegs;
+ for (const MachineOperand &MO : CurrMI->all_uses()) {
+ Register Reg = MO.getReg();
+ if (MO.readsReg() && Reg.isVirtual() && SeenRegs.insert(Reg).second)
+ retireVirtReg(Reg, SI);
+ }
+ }
+ }
+
// Add new registers or mask bits.
for (const auto &MO : CurrMI->all_defs()) {
Register Reg = MO.getReg();
diff --git a/llvm/lib/Target/AMDGPU/GCNRegPressure.h b/llvm/lib/Target/AMDGPU/GCNRegPressure.h
index c17f8f33489ea..7fb60107367de 100644
--- a/llvm/lib/Target/AMDGPU/GCNRegPressure.h
+++ b/llvm/lib/Target/AMDGPU/GCNRegPressure.h
@@ -450,8 +450,11 @@ class GCNDownwardRPTracker : public GCNRPTracker {
/// it is assumed that the tracker is using an externally managed iterator,
/// and advance* calls will not update the state of the iterator. In such
/// cases, the tracker will move to the state at the provided \p MI .
+ /// If \p RetireDeadUses is false the uses that die at the instruction are
+ /// kept live. Callers that do this must restore the tracked state afterwards.
void advanceToNext(MachineInstr *MI = nullptr,
- bool UseInternalIterator = true);
+ bool UseInternalIterator = true,
+ bool RetireDeadUses = true);
/// Move to the state at the next MI. \p returns false if reached end of
/// block. If \p UseInternalIterator is true, then internal iterators are used
diff --git a/llvm/lib/Target/AMDGPU/SIFormMemoryClauses.cpp b/llvm/lib/Target/AMDGPU/SIFormMemoryClauses.cpp
index 905c2afa9fcd8..c0c46be61576b 100644
--- a/llvm/lib/Target/AMDGPU/SIFormMemoryClauses.cpp
+++ b/llvm/lib/Target/AMDGPU/SIFormMemoryClauses.cpp
@@ -189,11 +189,12 @@ bool SIFormMemoryClausesImpl::canBundle(const MachineInstr &MI,
// bundled into a memory clause.
bool SIFormMemoryClausesImpl::checkPressure(const MachineInstr &MI,
GCNDownwardRPTracker &RPT) {
- // NB: skip advanceBeforeNext() call. Since all defs will be marked
- // early-clobber they will all stay alive at least to the end of the
- // clause. Therefor we should not decrease pressure even if load
+ // NB: skip advanceBeforeNext() call and keep the dying uses live. Since all
+ // defs will be marked early-clobber they will all stay alive at least to the
+ // end of the clause. Therefor we should not decrease pressure even if load
// pointer becomes dead and could otherwise be reused for destination.
- RPT.advanceToNext();
+ RPT.advanceToNext(/*MI=*/nullptr, /*UseInternalIterator=*/true,
+ /*RetireDeadUses=*/false);
GCNRegPressure MaxPressure = RPT.moveMaxPressure();
unsigned Occupancy = MaxPressure.getOccupancy(
*ST,
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/fshl.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/fshl.ll
index 7e4e9c6aeefe1..d5b230428fba5 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/fshl.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/fshl.ll
@@ -8157,88 +8157,88 @@ define <2 x i128> @v_fshl_v2i128(<2 x i128> %lhs, <2 x i128> %rhs, <2 x i128> %a
; GFX6-LABEL: v_fshl_v2i128:
; GFX6: ; %bb.0:
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-NEXT: v_and_b32_e32 v19, 0x7f, v16
-; GFX6-NEXT: v_not_b32_e32 v18, 63
-; GFX6-NEXT: v_sub_i32_e32 v23, vcc, 64, v19
-; GFX6-NEXT: v_add_i32_e32 v27, vcc, v19, v18
-; GFX6-NEXT: v_lshr_b64 v[23:24], v[0:1], v23
-; GFX6-NEXT: v_lshl_b64 v[25:26], v[2:3], v19
-; GFX6-NEXT: v_lshl_b64 v[21:22], v[0:1], v19
-; GFX6-NEXT: v_lshl_b64 v[0:1], v[0:1], v27
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
-; GFX6-NEXT: v_or_b32_e32 v19, v23, v25
-; GFX6-NEXT: v_or_b32_e32 v23, v24, v26
-; GFX6-NEXT: v_cndmask_b32_e32 v0, v0, v19, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v1, v1, v23, vcc
-; GFX6-NEXT: v_cndmask_b32_e64 v19, v0, v2, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e64 v23, v1, v3, s[4:5]
-; GFX6-NEXT: v_lshr_b64 v[0:1], v[8:9], 1
+; GFX6-NEXT: v_and_b32_e32 v25, 0x7f, v16
+; GFX6-NEXT: v_sub_i32_e32 v17, vcc, 64, v25
+; GFX6-NEXT: v_lshr_b64 v[17:18], v[0:1], v17
+; GFX6-NEXT: v_lshl_b64 v[21:22], v[2:3], v25
+; GFX6-NEXT: v_or_b32_e32 v26, v17, v21
+; GFX6-NEXT: v_or_b32_e32 v27, v18, v22
; GFX6-NEXT: v_mov_b32_e32 v17, 0x7f
-; GFX6-NEXT: v_lshlrev_b32_e32 v2, 31, v10
-; GFX6-NEXT: v_or_b32_e32 v1, v1, v2
-; GFX6-NEXT: v_lshr_b64 v[2:3], v[10:11], 1
-; GFX6-NEXT: v_bfi_b32 v10, v16, 0, v17
-; GFX6-NEXT: v_cndmask_b32_e32 v24, 0, v21, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v25, 0, v22, vcc
-; GFX6-NEXT: v_add_i32_e32 v16, vcc, v10, v18
-; GFX6-NEXT: v_sub_i32_e32 v21, vcc, 64, v10
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v10
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v10
-; GFX6-NEXT: v_lshr_b64 v[8:9], v[2:3], v10
-; GFX6-NEXT: v_lshr_b64 v[10:11], v[0:1], v10
-; GFX6-NEXT: v_lshl_b64 v[21:22], v[2:3], v21
-; GFX6-NEXT: v_lshr_b64 v[2:3], v[2:3], v16
-; GFX6-NEXT: v_or_b32_e32 v10, v10, v21
-; GFX6-NEXT: v_or_b32_e32 v11, v11, v22
-; GFX6-NEXT: v_cndmask_b32_e32 v2, v2, v10, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v3, v3, v11, vcc
-; GFX6-NEXT: v_and_b32_e32 v16, 0x7f, v20
-; GFX6-NEXT: v_cndmask_b32_e64 v0, v2, v0, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e64 v1, v3, v1, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e32 v2, 0, v8, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v3, 0, v9, vcc
-; GFX6-NEXT: v_sub_i32_e32 v10, vcc, 64, v16
-; GFX6-NEXT: v_lshr_b64 v[10:11], v[4:5], v10
-; GFX6-NEXT: v_lshl_b64 v[21:22], v[6:7], v16
-; GFX6-NEXT: v_or_b32_e32 v2, v19, v2
-; GFX6-NEXT: v_add_i32_e32 v19, vcc, v16, v18
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v16
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v16
-; GFX6-NEXT: v_lshl_b64 v[8:9], v[4:5], v16
-; GFX6-NEXT: v_or_b32_e32 v16, v10, v21
-; GFX6-NEXT: v_or_b32_e32 v21, v11, v22
-; GFX6-NEXT: v_lshl_b64 v[10:11], v[4:5], v19
-; GFX6-NEXT: v_cndmask_b32_e32 v4, 0, v8, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v5, 0, v9, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v8, v10, v16, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v9, v11, v21, vcc
-; GFX6-NEXT: v_cndmask_b32_e64 v6, v8, v6, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e64 v7, v9, v7, s[4:5]
-; GFX6-NEXT: v_lshr_b64 v[8:9], v[12:13], 1
-; GFX6-NEXT: v_lshlrev_b32_e32 v10, 31, v14
-; GFX6-NEXT: v_or_b32_e32 v9, v9, v10
-; GFX6-NEXT: v_lshr_b64 v[10:11], v[14:15], 1
-; GFX6-NEXT: v_bfi_b32 v14, v20, 0, v17
-; GFX6-NEXT: v_add_i32_e32 v18, vcc, v14, v18
-; GFX6-NEXT: v_sub_i32_e32 v16, vcc, 64, v14
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v14
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v14
-; GFX6-NEXT: v_lshr_b64 v[12:13], v[10:11], v14
-; GFX6-NEXT: v_lshr_b64 v[14:15], v[8:9], v14
-; GFX6-NEXT: v_lshl_b64 v[16:17], v[10:11], v16
-; GFX6-NEXT: v_lshr_b64 v[10:11], v[10:11], v18
+; GFX6-NEXT: v_lshr_b64 v[18:19], v[8:9], 1
+; GFX6-NEXT: v_lshlrev_b32_e32 v8, 31, v10
+; GFX6-NEXT: v_lshr_b64 v[9:10], v[10:11], 1
+; GFX6-NEXT: v_bfi_b32 v11, v16, 0, v17
+; GFX6-NEXT: v_or_b32_e32 v19, v19, v8
+; GFX6-NEXT: v_sub_i32_e32 v8, vcc, 64, v11
+; GFX6-NEXT: v_lshl_b64 v[21:22], v[9:10], v8
+; GFX6-NEXT: v_lshr_b64 v[23:24], v[18:19], v11
+; GFX6-NEXT: v_not_b32_e32 v8, 63
+; GFX6-NEXT: v_or_b32_e32 v16, v23, v21
+; GFX6-NEXT: v_add_i32_e32 v21, vcc, v25, v8
+; GFX6-NEXT: v_or_b32_e32 v23, v24, v22
+; GFX6-NEXT: v_lshl_b64 v[21:22], v[0:1], v21
+; GFX6-NEXT: v_lshl_b64 v[0:1], v[0:1], v25
+; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v25
+; GFX6-NEXT: v_cndmask_b32_e32 v24, 0, v0, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v0, v21, v26, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v21, v22, v27, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v22, 0, v1, vcc
+; GFX6-NEXT: v_cmp_eq_u32_e32 vcc, 0, v25
+; GFX6-NEXT: v_cndmask_b32_e32 v25, v0, v2, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v21, v21, v3, vcc
+; GFX6-NEXT: v_add_i32_e32 v0, vcc, v11, v8
+; GFX6-NEXT: v_lshr_b64 v[0:1], v[9:10], v0
+; GFX6-NEXT: v_lshr_b64 v[2:3], v[9:10], v11
+; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v11
+; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v11
+; GFX6-NEXT: v_and_b32_e32 v11, 0x7f, v20
+; GFX6-NEXT: v_cndmask_b32_e32 v0, v0, v16, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v1, v1, v23, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v3, 0, v3, vcc
+; GFX6-NEXT: v_sub_i32_e32 v9, vcc, 64, v11
+; GFX6-NEXT: v_cndmask_b32_e64 v0, v0, v18, s[4:5]
+; GFX6-NEXT: v_cndmask_b32_e64 v1, v1, v19, s[4:5]
+; GFX6-NEXT: v_lshr_b64 v[9:10], v[4:5], v9
+; GFX6-NEXT: v_lshl_b64 v[18:19], v[6:7], v11
+; GFX6-NEXT: v_add_i32_e32 v16, vcc, v11, v8
+; GFX6-NEXT: v_or_b32_e32 v1, v22, v1
+; GFX6-NEXT: v_or_b32_e32 v3, v21, v3
+; GFX6-NEXT: v_lshl_b64 v[21:22], v[4:5], v11
+; GFX6-NEXT: v_or_b32_e32 v18, v9, v18
+; GFX6-NEXT: v_or_b32_e32 v19, v10, v19
+; GFX6-NEXT: v_lshl_b64 v[9:10], v[4:5], v16
+; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v11
+; GFX6-NEXT: v_cndmask_b32_e32 v4, 0, v21, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v5, 0, v22, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v9, v9, v18, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v10, v10, v19, vcc
+; GFX6-NEXT: v_cmp_eq_u32_e32 vcc, 0, v11
+; GFX6-NEXT: v_cndmask_b32_e32 v6, v9, v6, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v7, v10, v7, vcc
+; GFX6-NEXT: v_lshr_b64 v[9:10], v[12:13], 1
+; GFX6-NEXT: v_lshlrev_b32_e32 v11, 31, v14
+; GFX6-NEXT: v_or_b32_e32 v10, v10, v11
+; GFX6-NEXT: v_lshr_b64 v[11:12], v[14:15], 1
+; GFX6-NEXT: v_bfi_b32 v19, v20, 0, v17
+; GFX6-NEXT: v_sub_i32_e32 v15, vcc, 64, v19
+; GFX6-NEXT: v_add_i32_e32 v8, vcc, v19, v8
+; GFX6-NEXT: v_lshr_b64 v[13:14], v[9:10], v19
+; GFX6-NEXT: v_lshl_b64 v[15:16], v[11:12], v15
+; GFX6-NEXT: v_lshr_b64 v[17:18], v[11:12], v19
+; GFX6-NEXT: v_lshr_b64 v[11:12], v[11:12], v8
+; GFX6-NEXT: v_or_b32_e32 v13, v13, v15
; GFX6-NEXT: v_or_b32_e32 v14, v14, v16
-; GFX6-NEXT: v_or_b32_e32 v15, v15, v17
-; GFX6-NEXT: v_cndmask_b32_e32 v10, v10, v14, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v11, v11, v15, vcc
-; GFX6-NEXT: v_cndmask_b32_e64 v8, v10, v8, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e64 v9, v11, v9, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e32 v10, 0, v12, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v11, 0, v13, vcc
+; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
+; GFX6-NEXT: v_cndmask_b32_e32 v8, v11, v13, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v11, v12, v14, vcc
+; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
+; GFX6-NEXT: v_cndmask_b32_e64 v8, v8, v9, s[4:5]
+; GFX6-NEXT: v_cndmask_b32_e64 v9, v11, v10, s[4:5]
+; GFX6-NEXT: v_cndmask_b32_e32 v10, 0, v17, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v11, 0, v18, vcc
; GFX6-NEXT: v_or_b32_e32 v0, v24, v0
-; GFX6-NEXT: v_or_b32_e32 v1, v25, v1
-; GFX6-NEXT: v_or_b32_e32 v3, v23, v3
+; GFX6-NEXT: v_or_b32_e32 v2, v25, v2
; GFX6-NEXT: v_or_b32_e32 v4, v4, v8
; GFX6-NEXT: v_or_b32_e32 v5, v5, v9
; GFX6-NEXT: v_or_b32_e32 v6, v6, v10
@@ -8248,88 +8248,88 @@ define <2 x i128> @v_fshl_v2i128(<2 x i128> %lhs, <2 x i128> %rhs, <2 x i128> %a
; GFX8-LABEL: v_fshl_v2i128:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-NEXT: v_and_b32_e32 v19, 0x7f, v16
-; GFX8-NEXT: v_not_b32_e32 v18, 63
-; GFX8-NEXT: v_sub_u32_e32 v23, vcc, 64, v19
-; GFX8-NEXT: v_add_u32_e32 v27, vcc, v19, v18
-; GFX8-NEXT: v_lshrrev_b64 v[23:24], v23, v[0:1]
-; GFX8-NEXT: v_lshlrev_b64 v[25:26], v19, v[2:3]
-; GFX8-NEXT: v_lshlrev_b64 v[21:22], v19, v[0:1]
-; GFX8-NEXT: v_lshlrev_b64 v[0:1], v27, v[0:1]
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
-; GFX8-NEXT: v_or_b32_e32 v19, v23, v25
-; GFX8-NEXT: v_or_b32_e32 v23, v24, v26
-; GFX8-NEXT: v_cndmask_b32_e32 v0, v0, v19, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v23, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v19, v0, v2, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v23, v1, v3, s[4:5]
-; GFX8-NEXT: v_lshrrev_b64 v[0:1], 1, v[8:9]
+; GFX8-NEXT: v_and_b32_e32 v25, 0x7f, v16
+; GFX8-NEXT: v_sub_u32_e32 v17, vcc, 64, v25
+; GFX8-NEXT: v_lshrrev_b64 v[17:18], v17, v[0:1]
+; GFX8-NEXT: v_lshlrev_b64 v[21:22], v25, v[2:3]
+; GFX8-NEXT: v_or_b32_e32 v26, v17, v21
+; GFX8-NEXT: v_or_b32_e32 v27, v18, v22
; GFX8-NEXT: v_mov_b32_e32 v17, 0x7f
-; GFX8-NEXT: v_lshlrev_b32_e32 v2, 31, v10
-; GFX8-NEXT: v_or_b32_e32 v1, v1, v2
-; GFX8-NEXT: v_lshrrev_b64 v[2:3], 1, v[10:11]
-; GFX8-NEXT: v_bfi_b32 v10, v16, 0, v17
-; GFX8-NEXT: v_cndmask_b32_e32 v24, 0, v21, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v25, 0, v22, vcc
-; GFX8-NEXT: v_add_u32_e32 v16, vcc, v10, v18
-; GFX8-NEXT: v_sub_u32_e32 v21, vcc, 64, v10
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v10
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v10
-; GFX8-NEXT: v_lshrrev_b64 v[8:9], v10, v[2:3]
-; GFX8-NEXT: v_lshrrev_b64 v[10:11], v10, v[0:1]
-; GFX8-NEXT: v_lshlrev_b64 v[21:22], v21, v[2:3]
-; GFX8-NEXT: v_lshrrev_b64 v[2:3], v16, v[2:3]
-; GFX8-NEXT: v_or_b32_e32 v10, v10, v21
-; GFX8-NEXT: v_or_b32_e32 v11, v11, v22
-; GFX8-NEXT: v_cndmask_b32_e32 v2, v2, v10, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v3, v11, vcc
-; GFX8-NEXT: v_and_b32_e32 v16, 0x7f, v20
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v2, v0, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v3, v1, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v2, 0, v8, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v3, 0, v9, vcc
-; GFX8-NEXT: v_sub_u32_e32 v10, vcc, 64, v16
-; GFX8-NEXT: v_lshrrev_b64 v[10:11], v10, v[4:5]
-; GFX8-NEXT: v_lshlrev_b64 v[21:22], v16, v[6:7]
-; GFX8-NEXT: v_or_b32_e32 v2, v19, v2
-; GFX8-NEXT: v_add_u32_e32 v19, vcc, v16, v18
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v16
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v16
-; GFX8-NEXT: v_lshlrev_b64 v[8:9], v16, v[4:5]
-; GFX8-NEXT: v_or_b32_e32 v16, v10, v21
-; GFX8-NEXT: v_or_b32_e32 v21, v11, v22
-; GFX8-NEXT: v_lshlrev_b64 v[10:11], v19, v[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v4, 0, v8, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v5, 0, v9, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v8, v10, v16, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v9, v11, v21, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v8, v6, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v9, v7, s[4:5]
-; GFX8-NEXT: v_lshrrev_b64 v[8:9], 1, v[12:13]
-; GFX8-NEXT: v_lshlrev_b32_e32 v10, 31, v14
-; GFX8-NEXT: v_or_b32_e32 v9, v9, v10
-; GFX8-NEXT: v_lshrrev_b64 v[10:11], 1, v[14:15]
-; GFX8-NEXT: v_bfi_b32 v14, v20, 0, v17
-; GFX8-NEXT: v_add_u32_e32 v18, vcc, v14, v18
-; GFX8-NEXT: v_sub_u32_e32 v16, vcc, 64, v14
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v14
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v14
-; GFX8-NEXT: v_lshrrev_b64 v[12:13], v14, v[10:11]
-; GFX8-NEXT: v_lshrrev_b64 v[14:15], v14, v[8:9]
-; GFX8-NEXT: v_lshlrev_b64 v[16:17], v16, v[10:11]
-; GFX8-NEXT: v_lshrrev_b64 v[10:11], v18, v[10:11]
+; GFX8-NEXT: v_lshrrev_b64 v[18:19], 1, v[8:9]
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 31, v10
+; GFX8-NEXT: v_lshrrev_b64 v[9:10], 1, v[10:11]
+; GFX8-NEXT: v_bfi_b32 v11, v16, 0, v17
+; GFX8-NEXT: v_or_b32_e32 v19, v19, v8
+; GFX8-NEXT: v_sub_u32_e32 v8, vcc, 64, v11
+; GFX8-NEXT: v_lshlrev_b64 v[21:22], v8, v[9:10]
+; GFX8-NEXT: v_lshrrev_b64 v[23:24], v11, v[18:19]
+; GFX8-NEXT: v_not_b32_e32 v8, 63
+; GFX8-NEXT: v_or_b32_e32 v16, v23, v21
+; GFX8-NEXT: v_add_u32_e32 v21, vcc, v25, v8
+; GFX8-NEXT: v_or_b32_e32 v23, v24, v22
+; GFX8-NEXT: v_lshlrev_b64 v[21:22], v21, v[0:1]
+; GFX8-NEXT: v_lshlrev_b64 v[0:1], v25, v[0:1]
+; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v25
+; GFX8-NEXT: v_cndmask_b32_e32 v24, 0, v0, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v0, v21, v26, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v21, v22, v27, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v22, 0, v1, vcc
+; GFX8-NEXT: v_cmp_eq_u32_e32 vcc, 0, v25
+; GFX8-NEXT: v_cndmask_b32_e32 v25, v0, v2, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v21, v21, v3, vcc
+; GFX8-NEXT: v_add_u32_e32 v0, vcc, v11, v8
+; GFX8-NEXT: v_lshrrev_b64 v[0:1], v0, v[9:10]
+; GFX8-NEXT: v_lshrrev_b64 v[2:3], v11, v[9:10]
+; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v11
+; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v11
+; GFX8-NEXT: v_and_b32_e32 v11, 0x7f, v20
+; GFX8-NEXT: v_cndmask_b32_e32 v0, v0, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v23, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v3, 0, v3, vcc
+; GFX8-NEXT: v_sub_u32_e32 v9, vcc, 64, v11
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, v18, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v19, s[4:5]
+; GFX8-NEXT: v_lshrrev_b64 v[9:10], v9, v[4:5]
+; GFX8-NEXT: v_lshlrev_b64 v[18:19], v11, v[6:7]
+; GFX8-NEXT: v_add_u32_e32 v16, vcc, v11, v8
+; GFX8-NEXT: v_or_b32_e32 v1, v22, v1
+; GFX8-NEXT: v_or_b32_e32 v3, v21, v3
+; GFX8-NEXT: v_lshlrev_b64 v[21:22], v11, v[4:5]
+; GFX8-NEXT: v_or_b32_e32 v18, v9, v18
+; GFX8-NEXT: v_or_b32_e32 v19, v10, v19
+; GFX8-NEXT: v_lshlrev_b64 v[9:10], v16, v[4:5]
+; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v11
+; GFX8-NEXT: v_cndmask_b32_e32 v4, 0, v21, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v5, 0, v22, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v9, v18, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v10, v10, v19, vcc
+; GFX8-NEXT: v_cmp_eq_u32_e32 vcc, 0, v11
+; GFX8-NEXT: v_cndmask_b32_e32 v6, v9, v6, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v7, v10, v7, vcc
+; GFX8-NEXT: v_lshrrev_b64 v[9:10], 1, v[12:13]
+; GFX8-NEXT: v_lshlrev_b32_e32 v11, 31, v14
+; GFX8-NEXT: v_or_b32_e32 v10, v10, v11
+; GFX8-NEXT: v_lshrrev_b64 v[11:12], 1, v[14:15]
+; GFX8-NEXT: v_bfi_b32 v19, v20, 0, v17
+; GFX8-NEXT: v_sub_u32_e32 v15, vcc, 64, v19
+; GFX8-NEXT: v_add_u32_e32 v8, vcc, v19, v8
+; GFX8-NEXT: v_lshrrev_b64 v[13:14], v19, v[9:10]
+; GFX8-NEXT: v_lshlrev_b64 v[15:16], v15, v[11:12]
+; GFX8-NEXT: v_lshrrev_b64 v[17:18], v19, v[11:12]
+; GFX8-NEXT: v_lshrrev_b64 v[11:12], v8, v[11:12]
+; GFX8-NEXT: v_or_b32_e32 v13, v13, v15
; GFX8-NEXT: v_or_b32_e32 v14, v14, v16
-; GFX8-NEXT: v_or_b32_e32 v15, v15, v17
-; GFX8-NEXT: v_cndmask_b32_e32 v10, v10, v14, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v11, v11, v15, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v10, v8, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v11, v9, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v10, 0, v12, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v11, 0, v13, vcc
+; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
+; GFX8-NEXT: v_cndmask_b32_e32 v8, v11, v13, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v11, v12, v14, vcc
+; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
+; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, v9, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v9, v11, v10, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e32 v10, 0, v17, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v11, 0, v18, vcc
; GFX8-NEXT: v_or_b32_e32 v0, v24, v0
-; GFX8-NEXT: v_or_b32_e32 v1, v25, v1
-; GFX8-NEXT: v_or_b32_e32 v3, v23, v3
+; GFX8-NEXT: v_or_b32_e32 v2, v25, v2
; GFX8-NEXT: v_or_b32_e32 v4, v4, v8
; GFX8-NEXT: v_or_b32_e32 v5, v5, v9
; GFX8-NEXT: v_or_b32_e32 v6, v6, v10
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/fshr.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/fshr.ll
index 6ae2b9f9e8e1b..2cb9ac53004bf 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/fshr.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/fshr.ll
@@ -7863,182 +7863,182 @@ define <2 x i128> @v_fshr_v2i128(<2 x i128> %lhs, <2 x i128> %rhs, <2 x i128> %a
; GFX6: ; %bb.0:
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX6-NEXT: v_lshl_b64 v[2:3], v[2:3], 1
-; GFX6-NEXT: v_mov_b32_e32 v18, 0x7f
-; GFX6-NEXT: v_lshl_b64 v[21:22], v[0:1], 1
+; GFX6-NEXT: v_mov_b32_e32 v19, 0x7f
+; GFX6-NEXT: v_lshl_b64 v[17:18], v[0:1], 1
; GFX6-NEXT: v_lshrrev_b32_e32 v0, 31, v1
-; GFX6-NEXT: v_bfi_b32 v19, v16, 0, v18
+; GFX6-NEXT: v_bfi_b32 v23, v16, 0, v19
; GFX6-NEXT: v_or_b32_e32 v2, v2, v0
-; GFX6-NEXT: v_not_b32_e32 v17, 63
-; GFX6-NEXT: v_sub_i32_e32 v23, vcc, 64, v19
-; GFX6-NEXT: v_add_i32_e32 v27, vcc, v19, v17
-; GFX6-NEXT: v_lshr_b64 v[23:24], v[21:22], v23
-; GFX6-NEXT: v_lshl_b64 v[25:26], v[2:3], v19
-; GFX6-NEXT: v_lshl_b64 v[0:1], v[21:22], v19
-; GFX6-NEXT: v_lshl_b64 v[21:22], v[21:22], v27
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
-; GFX6-NEXT: v_or_b32_e32 v19, v23, v25
-; GFX6-NEXT: v_or_b32_e32 v23, v24, v26
-; GFX6-NEXT: v_cndmask_b32_e32 v24, 0, v0, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v0, v21, v19, vcc
-; GFX6-NEXT: v_cndmask_b32_e64 v19, v0, v2, s[4:5]
-; GFX6-NEXT: v_and_b32_e32 v2, 0x7f, v16
-; GFX6-NEXT: v_cndmask_b32_e32 v25, 0, v1, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v1, v22, v23, vcc
-; GFX6-NEXT: v_add_i32_e32 v16, vcc, v2, v17
-; GFX6-NEXT: v_sub_i32_e32 v21, vcc, 64, v2
-; GFX6-NEXT: v_cndmask_b32_e64 v23, v1, v3, s[4:5]
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v2
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v2
-; GFX6-NEXT: v_lshr_b64 v[0:1], v[10:11], v2
-; GFX6-NEXT: v_lshr_b64 v[2:3], v[8:9], v2
-; GFX6-NEXT: v_lshl_b64 v[21:22], v[10:11], v21
+; GFX6-NEXT: v_sub_i32_e32 v0, vcc, 64, v23
+; GFX6-NEXT: v_lshr_b64 v[0:1], v[17:18], v0
+; GFX6-NEXT: v_lshl_b64 v[21:22], v[2:3], v23
+; GFX6-NEXT: v_and_b32_e32 v26, 0x7f, v16
+; GFX6-NEXT: v_or_b32_e32 v24, v0, v21
+; GFX6-NEXT: v_sub_i32_e32 v0, vcc, 64, v26
+; GFX6-NEXT: v_or_b32_e32 v25, v1, v22
+; GFX6-NEXT: v_lshl_b64 v[0:1], v[10:11], v0
+; GFX6-NEXT: v_lshr_b64 v[21:22], v[8:9], v26
+; GFX6-NEXT: v_not_b32_e32 v27, 63
+; GFX6-NEXT: v_or_b32_e32 v21, v21, v0
+; GFX6-NEXT: v_add_i32_e32 v0, vcc, v23, v27
+; GFX6-NEXT: v_or_b32_e32 v22, v22, v1
+; GFX6-NEXT: v_lshl_b64 v[0:1], v[17:18], v0
+; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v23
+; GFX6-NEXT: v_cndmask_b32_e32 v0, v0, v24, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v1, v1, v25, vcc
+; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v23
+; GFX6-NEXT: v_cndmask_b32_e64 v2, v0, v2, s[4:5]
+; GFX6-NEXT: v_cndmask_b32_e64 v3, v1, v3, s[4:5]
+; GFX6-NEXT: v_add_i32_e64 v0, s[4:5], v26, v27
+; GFX6-NEXT: v_lshr_b64 v[0:1], v[10:11], v0
+; GFX6-NEXT: v_cmp_gt_u32_e64 s[4:5], 64, v26
+; GFX6-NEXT: v_lshl_b64 v[16:17], v[17:18], v23
+; GFX6-NEXT: v_cndmask_b32_e64 v0, v0, v21, s[4:5]
+; GFX6-NEXT: v_cndmask_b32_e64 v1, v1, v22, s[4:5]
+; GFX6-NEXT: v_cmp_eq_u32_e64 s[6:7], 0, v26
+; GFX6-NEXT: v_cndmask_b32_e64 v8, v0, v8, s[6:7]
+; GFX6-NEXT: v_cndmask_b32_e64 v9, v1, v9, s[6:7]
+; GFX6-NEXT: v_lshr_b64 v[0:1], v[10:11], v26
+; GFX6-NEXT: v_cndmask_b32_e32 v16, 0, v16, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v10, 0, v17, vcc
; GFX6-NEXT: v_lshl_b64 v[6:7], v[6:7], 1
-; GFX6-NEXT: v_or_b32_e32 v21, v2, v21
-; GFX6-NEXT: v_or_b32_e32 v22, v3, v22
-; GFX6-NEXT: v_lshr_b64 v[2:3], v[10:11], v16
-; GFX6-NEXT: v_bfi_b32 v16, v20, 0, v18
-; GFX6-NEXT: v_cndmask_b32_e32 v2, v2, v21, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v3, v3, v22, vcc
-; GFX6-NEXT: v_cndmask_b32_e64 v2, v2, v8, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e64 v3, v3, v9, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e32 v8, 0, v0, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v9, 0, v1, vcc
-; GFX6-NEXT: v_or_b32_e32 v0, v24, v2
-; GFX6-NEXT: v_or_b32_e32 v1, v25, v3
-; GFX6-NEXT: v_or_b32_e32 v2, v19, v8
-; GFX6-NEXT: v_or_b32_e32 v3, v23, v9
+; GFX6-NEXT: v_cndmask_b32_e64 v11, 0, v0, s[4:5]
+; GFX6-NEXT: v_cndmask_b32_e64 v17, 0, v1, s[4:5]
+; GFX6-NEXT: v_or_b32_e32 v0, v16, v8
+; GFX6-NEXT: v_or_b32_e32 v1, v10, v9
; GFX6-NEXT: v_lshl_b64 v[8:9], v[4:5], 1
; GFX6-NEXT: v_lshrrev_b32_e32 v4, 31, v5
+; GFX6-NEXT: v_bfi_b32 v16, v20, 0, v19
; GFX6-NEXT: v_or_b32_e32 v6, v6, v4
-; GFX6-NEXT: v_sub_i32_e32 v10, vcc, 64, v16
-; GFX6-NEXT: v_add_i32_e32 v21, vcc, v16, v17
-; GFX6-NEXT: v_lshr_b64 v[10:11], v[8:9], v10
-; GFX6-NEXT: v_lshl_b64 v[18:19], v[6:7], v16
+; GFX6-NEXT: v_sub_i32_e32 v4, vcc, 64, v16
+; GFX6-NEXT: v_or_b32_e32 v2, v2, v11
+; GFX6-NEXT: v_lshr_b64 v[4:5], v[8:9], v4
+; GFX6-NEXT: v_lshl_b64 v[10:11], v[6:7], v16
+; GFX6-NEXT: v_or_b32_e32 v3, v3, v17
+; GFX6-NEXT: v_add_i32_e32 v17, vcc, v16, v27
+; GFX6-NEXT: v_or_b32_e32 v10, v4, v10
+; GFX6-NEXT: v_or_b32_e32 v11, v5, v11
; GFX6-NEXT: v_lshl_b64 v[4:5], v[8:9], v16
-; GFX6-NEXT: v_lshl_b64 v[8:9], v[8:9], v21
+; GFX6-NEXT: v_lshl_b64 v[8:9], v[8:9], v17
; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v16
-; GFX6-NEXT: v_or_b32_e32 v10, v10, v18
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v16
-; GFX6-NEXT: v_cndmask_b32_e32 v16, 0, v4, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v4, v8, v10, vcc
-; GFX6-NEXT: v_or_b32_e32 v11, v11, v19
-; GFX6-NEXT: v_cndmask_b32_e64 v10, v4, v6, s[4:5]
-; GFX6-NEXT: v_and_b32_e32 v6, 0x7f, v20
+; GFX6-NEXT: v_cndmask_b32_e32 v17, 0, v4, vcc
; GFX6-NEXT: v_cndmask_b32_e32 v18, 0, v5, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v4, v8, v10, vcc
; GFX6-NEXT: v_cndmask_b32_e32 v5, v9, v11, vcc
-; GFX6-NEXT: v_add_i32_e32 v17, vcc, v6, v17
-; GFX6-NEXT: v_sub_i32_e32 v8, vcc, 64, v6
-; GFX6-NEXT: v_cndmask_b32_e64 v11, v5, v7, s[4:5]
-; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v6
-; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v6
-; GFX6-NEXT: v_lshr_b64 v[4:5], v[14:15], v6
-; GFX6-NEXT: v_lshr_b64 v[6:7], v[12:13], v6
-; GFX6-NEXT: v_lshl_b64 v[8:9], v[14:15], v8
-; GFX6-NEXT: v_or_b32_e32 v8, v6, v8
-; GFX6-NEXT: v_or_b32_e32 v9, v7, v9
-; GFX6-NEXT: v_lshr_b64 v[6:7], v[14:15], v17
-; GFX6-NEXT: v_cndmask_b32_e32 v6, v6, v8, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v7, v7, v9, vcc
+; GFX6-NEXT: v_cmp_eq_u32_e32 vcc, 0, v16
+; GFX6-NEXT: v_and_b32_e32 v10, 0x7f, v20
+; GFX6-NEXT: v_cndmask_b32_e32 v8, v4, v6, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v9, v5, v7, vcc
+; GFX6-NEXT: v_sub_i32_e32 v6, vcc, 64, v10
+; GFX6-NEXT: v_lshr_b64 v[4:5], v[12:13], v10
+; GFX6-NEXT: v_lshl_b64 v[6:7], v[14:15], v6
+; GFX6-NEXT: v_add_i32_e32 v11, vcc, v10, v27
+; GFX6-NEXT: v_or_b32_e32 v16, v4, v6
+; GFX6-NEXT: v_or_b32_e32 v19, v5, v7
+; GFX6-NEXT: v_lshr_b64 v[6:7], v[14:15], v11
+; GFX6-NEXT: v_lshr_b64 v[4:5], v[14:15], v10
+; GFX6-NEXT: v_cmp_gt_u32_e32 vcc, 64, v10
+; GFX6-NEXT: v_cndmask_b32_e32 v6, v6, v16, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v7, v7, v19, vcc
+; GFX6-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v10
; GFX6-NEXT: v_cndmask_b32_e64 v6, v6, v12, s[4:5]
; GFX6-NEXT: v_cndmask_b32_e64 v7, v7, v13, s[4:5]
-; GFX6-NEXT: v_cndmask_b32_e32 v8, 0, v4, vcc
-; GFX6-NEXT: v_cndmask_b32_e32 v9, 0, v5, vcc
-; GFX6-NEXT: v_or_b32_e32 v4, v16, v6
+; GFX6-NEXT: v_cndmask_b32_e32 v10, 0, v4, vcc
+; GFX6-NEXT: v_cndmask_b32_e32 v11, 0, v5, vcc
+; GFX6-NEXT: v_or_b32_e32 v4, v17, v6
; GFX6-NEXT: v_or_b32_e32 v5, v18, v7
-; GFX6-NEXT: v_or_b32_e32 v6, v10, v8
-; GFX6-NEXT: v_or_b32_e32 v7, v11, v9
+; GFX6-NEXT: v_or_b32_e32 v6, v8, v10
+; GFX6-NEXT: v_or_b32_e32 v7, v9, v11
; GFX6-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: v_fshr_v2i128:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-NEXT: v_lshlrev_b64 v[2:3], 1, v[2:3]
-; GFX8-NEXT: v_mov_b32_e32 v18, 0x7f
-; GFX8-NEXT: v_lshlrev_b64 v[21:22], 1, v[0:1]
+; GFX8-NEXT: v_mov_b32_e32 v19, 0x7f
+; GFX8-NEXT: v_lshlrev_b64 v[17:18], 1, v[0:1]
; GFX8-NEXT: v_lshrrev_b32_e32 v0, 31, v1
-; GFX8-NEXT: v_bfi_b32 v19, v16, 0, v18
+; GFX8-NEXT: v_bfi_b32 v23, v16, 0, v19
; GFX8-NEXT: v_or_b32_e32 v2, v2, v0
-; GFX8-NEXT: v_not_b32_e32 v17, 63
-; GFX8-NEXT: v_sub_u32_e32 v23, vcc, 64, v19
-; GFX8-NEXT: v_add_u32_e32 v27, vcc, v19, v17
-; GFX8-NEXT: v_lshrrev_b64 v[23:24], v23, v[21:22]
-; GFX8-NEXT: v_lshlrev_b64 v[25:26], v19, v[2:3]
-; GFX8-NEXT: v_lshlrev_b64 v[0:1], v19, v[21:22]
-; GFX8-NEXT: v_lshlrev_b64 v[21:22], v27, v[21:22]
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
-; GFX8-NEXT: v_or_b32_e32 v19, v23, v25
-; GFX8-NEXT: v_or_b32_e32 v23, v24, v26
-; GFX8-NEXT: v_cndmask_b32_e32 v24, 0, v0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v0, v21, v19, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v19, v0, v2, s[4:5]
-; GFX8-NEXT: v_and_b32_e32 v2, 0x7f, v16
-; GFX8-NEXT: v_cndmask_b32_e32 v25, 0, v1, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v22, v23, vcc
-; GFX8-NEXT: v_add_u32_e32 v16, vcc, v2, v17
-; GFX8-NEXT: v_sub_u32_e32 v21, vcc, 64, v2
-; GFX8-NEXT: v_cndmask_b32_e64 v23, v1, v3, s[4:5]
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v2
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v2
-; GFX8-NEXT: v_lshrrev_b64 v[0:1], v2, v[10:11]
-; GFX8-NEXT: v_lshrrev_b64 v[2:3], v2, v[8:9]
-; GFX8-NEXT: v_lshlrev_b64 v[21:22], v21, v[10:11]
+; GFX8-NEXT: v_sub_u32_e32 v0, vcc, 64, v23
+; GFX8-NEXT: v_lshrrev_b64 v[0:1], v0, v[17:18]
+; GFX8-NEXT: v_lshlrev_b64 v[21:22], v23, v[2:3]
+; GFX8-NEXT: v_and_b32_e32 v26, 0x7f, v16
+; GFX8-NEXT: v_or_b32_e32 v24, v0, v21
+; GFX8-NEXT: v_sub_u32_e32 v0, vcc, 64, v26
+; GFX8-NEXT: v_or_b32_e32 v25, v1, v22
+; GFX8-NEXT: v_lshlrev_b64 v[0:1], v0, v[10:11]
+; GFX8-NEXT: v_lshrrev_b64 v[21:22], v26, v[8:9]
+; GFX8-NEXT: v_not_b32_e32 v27, 63
+; GFX8-NEXT: v_or_b32_e32 v21, v21, v0
+; GFX8-NEXT: v_add_u32_e32 v0, vcc, v23, v27
+; GFX8-NEXT: v_or_b32_e32 v22, v22, v1
+; GFX8-NEXT: v_lshlrev_b64 v[0:1], v0, v[17:18]
+; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v23
+; GFX8-NEXT: v_cndmask_b32_e32 v0, v0, v24, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v25, vcc
+; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v23
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v0, v2, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v1, v3, s[4:5]
+; GFX8-NEXT: v_add_u32_e64 v0, s[4:5], v26, v27
+; GFX8-NEXT: v_lshrrev_b64 v[0:1], v0, v[10:11]
+; GFX8-NEXT: v_cmp_gt_u32_e64 s[4:5], 64, v26
+; GFX8-NEXT: v_lshlrev_b64 v[16:17], v23, v[17:18]
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, v21, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v22, s[4:5]
+; GFX8-NEXT: v_cmp_eq_u32_e64 s[6:7], 0, v26
+; GFX8-NEXT: v_cndmask_b32_e64 v8, v0, v8, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v9, v1, v9, s[6:7]
+; GFX8-NEXT: v_lshrrev_b64 v[0:1], v26, v[10:11]
+; GFX8-NEXT: v_cndmask_b32_e32 v16, 0, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v10, 0, v17, vcc
; GFX8-NEXT: v_lshlrev_b64 v[6:7], 1, v[6:7]
-; GFX8-NEXT: v_or_b32_e32 v21, v2, v21
-; GFX8-NEXT: v_or_b32_e32 v22, v3, v22
-; GFX8-NEXT: v_lshrrev_b64 v[2:3], v16, v[10:11]
-; GFX8-NEXT: v_bfi_b32 v16, v20, 0, v18
-; GFX8-NEXT: v_cndmask_b32_e32 v2, v2, v21, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v3, v22, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, v8, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v9, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v8, 0, v0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v9, 0, v1, vcc
-; GFX8-NEXT: v_or_b32_e32 v0, v24, v2
-; GFX8-NEXT: v_or_b32_e32 v1, v25, v3
-; GFX8-NEXT: v_or_b32_e32 v2, v19, v8
-; GFX8-NEXT: v_or_b32_e32 v3, v23, v9
+; GFX8-NEXT: v_cndmask_b32_e64 v11, 0, v0, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v17, 0, v1, s[4:5]
+; GFX8-NEXT: v_or_b32_e32 v0, v16, v8
+; GFX8-NEXT: v_or_b32_e32 v1, v10, v9
; GFX8-NEXT: v_lshlrev_b64 v[8:9], 1, v[4:5]
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 31, v5
+; GFX8-NEXT: v_bfi_b32 v16, v20, 0, v19
; GFX8-NEXT: v_or_b32_e32 v6, v6, v4
-; GFX8-NEXT: v_sub_u32_e32 v10, vcc, 64, v16
-; GFX8-NEXT: v_add_u32_e32 v21, vcc, v16, v17
-; GFX8-NEXT: v_lshrrev_b64 v[10:11], v10, v[8:9]
-; GFX8-NEXT: v_lshlrev_b64 v[18:19], v16, v[6:7]
+; GFX8-NEXT: v_sub_u32_e32 v4, vcc, 64, v16
+; GFX8-NEXT: v_or_b32_e32 v2, v2, v11
+; GFX8-NEXT: v_lshrrev_b64 v[4:5], v4, v[8:9]
+; GFX8-NEXT: v_lshlrev_b64 v[10:11], v16, v[6:7]
+; GFX8-NEXT: v_or_b32_e32 v3, v3, v17
+; GFX8-NEXT: v_add_u32_e32 v17, vcc, v16, v27
+; GFX8-NEXT: v_or_b32_e32 v10, v4, v10
+; GFX8-NEXT: v_or_b32_e32 v11, v5, v11
; GFX8-NEXT: v_lshlrev_b64 v[4:5], v16, v[8:9]
-; GFX8-NEXT: v_lshlrev_b64 v[8:9], v21, v[8:9]
+; GFX8-NEXT: v_lshlrev_b64 v[8:9], v17, v[8:9]
; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v16
-; GFX8-NEXT: v_or_b32_e32 v10, v10, v18
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v16
-; GFX8-NEXT: v_cndmask_b32_e32 v16, 0, v4, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v4, v8, v10, vcc
-; GFX8-NEXT: v_or_b32_e32 v11, v11, v19
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v4, v6, s[4:5]
-; GFX8-NEXT: v_and_b32_e32 v6, 0x7f, v20
+; GFX8-NEXT: v_cndmask_b32_e32 v17, 0, v4, vcc
; GFX8-NEXT: v_cndmask_b32_e32 v18, 0, v5, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v4, v8, v10, vcc
; GFX8-NEXT: v_cndmask_b32_e32 v5, v9, v11, vcc
-; GFX8-NEXT: v_add_u32_e32 v17, vcc, v6, v17
-; GFX8-NEXT: v_sub_u32_e32 v8, vcc, 64, v6
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v5, v7, s[4:5]
-; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v6
-; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v6
-; GFX8-NEXT: v_lshrrev_b64 v[4:5], v6, v[14:15]
-; GFX8-NEXT: v_lshrrev_b64 v[6:7], v6, v[12:13]
-; GFX8-NEXT: v_lshlrev_b64 v[8:9], v8, v[14:15]
-; GFX8-NEXT: v_or_b32_e32 v8, v6, v8
-; GFX8-NEXT: v_or_b32_e32 v9, v7, v9
-; GFX8-NEXT: v_lshrrev_b64 v[6:7], v17, v[14:15]
-; GFX8-NEXT: v_cndmask_b32_e32 v6, v6, v8, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v7, v7, v9, vcc
+; GFX8-NEXT: v_cmp_eq_u32_e32 vcc, 0, v16
+; GFX8-NEXT: v_and_b32_e32 v10, 0x7f, v20
+; GFX8-NEXT: v_cndmask_b32_e32 v8, v4, v6, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v5, v7, vcc
+; GFX8-NEXT: v_sub_u32_e32 v6, vcc, 64, v10
+; GFX8-NEXT: v_lshrrev_b64 v[4:5], v10, v[12:13]
+; GFX8-NEXT: v_lshlrev_b64 v[6:7], v6, v[14:15]
+; GFX8-NEXT: v_add_u32_e32 v11, vcc, v10, v27
+; GFX8-NEXT: v_or_b32_e32 v16, v4, v6
+; GFX8-NEXT: v_or_b32_e32 v19, v5, v7
+; GFX8-NEXT: v_lshrrev_b64 v[6:7], v11, v[14:15]
+; GFX8-NEXT: v_lshrrev_b64 v[4:5], v10, v[14:15]
+; GFX8-NEXT: v_cmp_gt_u32_e32 vcc, 64, v10
+; GFX8-NEXT: v_cndmask_b32_e32 v6, v6, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v7, v7, v19, vcc
+; GFX8-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v10
; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, v12, s[4:5]
; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v13, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v8, 0, v4, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v9, 0, v5, vcc
-; GFX8-NEXT: v_or_b32_e32 v4, v16, v6
+; GFX8-NEXT: v_cndmask_b32_e32 v10, 0, v4, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v11, 0, v5, vcc
+; GFX8-NEXT: v_or_b32_e32 v4, v17, v6
; GFX8-NEXT: v_or_b32_e32 v5, v18, v7
-; GFX8-NEXT: v_or_b32_e32 v6, v10, v8
-; GFX8-NEXT: v_or_b32_e32 v7, v11, v9
+; GFX8-NEXT: v_or_b32_e32 v6, v8, v10
+; GFX8-NEXT: v_or_b32_e32 v7, v9, v11
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX9-LABEL: v_fshr_v2i128:
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/uaddsat.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/uaddsat.ll
index 8b77812f28f9c..e552f037a950b 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/uaddsat.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/uaddsat.ll
@@ -1653,10 +1653,10 @@ define <16 x i32> @v_uaddsat_v16i32(<16 x i32> %lhs, <16 x i32> %rhs) {
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX6-NEXT: v_not_b32_e32 v31, v0
; GFX6-NEXT: v_min_u32_e32 v16, v31, v16
-; GFX6-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX6-NEXT: v_add_i32_e32 v0, vcc, v0, v16
; GFX6-NEXT: v_not_b32_e32 v16, v1
; GFX6-NEXT: v_min_u32_e32 v16, v16, v17
+; GFX6-NEXT: buffer_load_dword v17, off, s[0:3], s32
; GFX6-NEXT: v_add_i32_e32 v1, vcc, v1, v16
; GFX6-NEXT: v_not_b32_e32 v16, v2
; GFX6-NEXT: v_min_u32_e32 v16, v16, v18
@@ -1699,7 +1699,7 @@ define <16 x i32> @v_uaddsat_v16i32(<16 x i32> %lhs, <16 x i32> %rhs) {
; GFX6-NEXT: v_add_i32_e32 v14, vcc, v14, v16
; GFX6-NEXT: v_not_b32_e32 v16, v15
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: v_min_u32_e32 v16, v16, v31
+; GFX6-NEXT: v_min_u32_e32 v16, v16, v17
; GFX6-NEXT: v_add_i32_e32 v15, vcc, v15, v16
; GFX6-NEXT: s_setpc_b64 s[30:31]
;
diff --git a/llvm/test/CodeGen/AMDGPU/addrspacecast.ll b/llvm/test/CodeGen/AMDGPU/addrspacecast.ll
index 18ed68396fa4f..d98f8fe42d1c0 100644
--- a/llvm/test/CodeGen/AMDGPU/addrspacecast.ll
+++ b/llvm/test/CodeGen/AMDGPU/addrspacecast.ll
@@ -736,130 +736,124 @@ define <16 x ptr> @addrspacecast_v16p5_to_v16p0(<16 x ptr addrspace(5)> %ptr) {
; CI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; CI-NEXT: s_load_dword s4, s[6:7], 0x11
; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v0
-; CI-NEXT: v_cndmask_b32_e32 v0, 0, v0, vcc
-; CI-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v5
-; CI-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v6
+; CI-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v2
+; CI-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v3
+; CI-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v4
; CI-NEXT: s_waitcnt lgkmcnt(0)
; CI-NEXT: v_mov_b32_e32 v31, s4
-; CI-NEXT: v_cndmask_b32_e32 v49, 0, v31, vcc
-; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v1
-; CI-NEXT: v_cndmask_b32_e32 v34, 0, v1, vcc
-; CI-NEXT: v_cndmask_b32_e32 v39, 0, v31, vcc
-; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v2
-; CI-NEXT: v_cndmask_b32_e32 v35, 0, v2, vcc
-; CI-NEXT: v_cndmask_b32_e32 v32, 0, v31, vcc
-; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v3
-; CI-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v4
-; CI-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v7
-; CI-NEXT: v_cndmask_b32_e32 v36, 0, v3, vcc
-; CI-NEXT: v_cndmask_b32_e64 v48, 0, v4, s[4:5]
-; CI-NEXT: v_cndmask_b32_e64 v37, 0, v5, s[6:7]
-; CI-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[8:9]
-; CI-NEXT: v_cndmask_b32_e64 v38, 0, v7, s[10:11]
-; CI-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v8
-; CI-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v9
-; CI-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v10
-; CI-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v11
-; CI-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v12
-; CI-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v13
-; CI-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v14
-; CI-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v15
-; CI-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[12:13]
-; CI-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[14:15]
-; CI-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[16:17]
-; CI-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[18:19]
-; CI-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[20:21]
-; CI-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[22:23]
-; CI-NEXT: v_cndmask_b32_e64 v28, 0, v14, s[24:25]
-; CI-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[26:27]
-; CI-NEXT: v_cndmask_b32_e32 v7, 0, v31, vcc
-; CI-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[4:5]
-; CI-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[6:7]
-; CI-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[8:9]
-; CI-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[10:11]
-; CI-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[12:13]
-; CI-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[14:15]
-; CI-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[16:17]
-; CI-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[18:19]
-; CI-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[20:21]
-; CI-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[22:23]
-; CI-NEXT: v_cndmask_b32_e64 v29, 0, v31, s[24:25]
-; CI-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[26:27]
-; CI-NEXT: v_mov_b32_e32 v1, v49
+; CI-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v1
+; CI-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v5
+; CI-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v6
+; CI-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v7
+; CI-NEXT: v_cndmask_b32_e32 v0, 0, v0, vcc
+; CI-NEXT: v_cndmask_b32_e64 v34, 0, v1, s[4:5]
+; CI-NEXT: v_cndmask_b32_e64 v38, 0, v2, s[6:7]
+; CI-NEXT: v_cndmask_b32_e64 v36, 0, v3, s[8:9]
+; CI-NEXT: v_cndmask_b32_e64 v32, 0, v4, s[10:11]
+; CI-NEXT: v_cndmask_b32_e64 v35, 0, v5, s[12:13]
+; CI-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[14:15]
+; CI-NEXT: v_cndmask_b32_e64 v37, 0, v7, s[16:17]
+; CI-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v8
+; CI-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v9
+; CI-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v10
+; CI-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v11
+; CI-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v12
+; CI-NEXT: v_cmp_ne_u32_e64 s[28:29], -1, v13
+; CI-NEXT: v_cndmask_b32_e32 v1, 0, v31, vcc
+; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v14
+; CI-NEXT: v_cmp_ne_u32_e64 s[40:41], -1, v15
+; CI-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[18:19]
+; CI-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[20:21]
+; CI-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[22:23]
+; CI-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[24:25]
+; CI-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[26:27]
+; CI-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[28:29]
+; CI-NEXT: v_cndmask_b32_e32 v28, 0, v14, vcc
+; CI-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[40:41]
+; CI-NEXT: v_cndmask_b32_e64 v3, 0, v31, s[4:5]
+; CI-NEXT: v_cndmask_b32_e64 v5, 0, v31, s[6:7]
+; CI-NEXT: v_cndmask_b32_e64 v7, 0, v31, s[8:9]
+; CI-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[10:11]
+; CI-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[12:13]
+; CI-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[14:15]
+; CI-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[16:17]
+; CI-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[18:19]
+; CI-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[20:21]
+; CI-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[22:23]
+; CI-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[24:25]
+; CI-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[26:27]
+; CI-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[28:29]
+; CI-NEXT: v_cndmask_b32_e32 v29, 0, v31, vcc
+; CI-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[40:41]
; CI-NEXT: v_mov_b32_e32 v2, v34
-; CI-NEXT: v_mov_b32_e32 v3, v39
-; CI-NEXT: v_mov_b32_e32 v4, v35
-; CI-NEXT: v_mov_b32_e32 v5, v32
+; CI-NEXT: v_mov_b32_e32 v4, v38
; CI-NEXT: v_mov_b32_e32 v6, v36
-; CI-NEXT: v_mov_b32_e32 v8, v48
-; CI-NEXT: v_mov_b32_e32 v10, v37
+; CI-NEXT: v_mov_b32_e32 v8, v32
+; CI-NEXT: v_mov_b32_e32 v10, v35
; CI-NEXT: v_mov_b32_e32 v12, v33
-; CI-NEXT: v_mov_b32_e32 v14, v38
+; CI-NEXT: v_mov_b32_e32 v14, v37
; CI-NEXT: s_setpc_b64 s[30:31]
;
; GFX9-LABEL: addrspacecast_v16p5_to_v16p0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX9-NEXT: s_mov_b64 s[4:5], src_private_base
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v0
; GFX9-NEXT: v_mov_b32_e32 v31, s5
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v1
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v2
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v3
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v4
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v5
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v6
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v7
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v0
+; GFX9-NEXT: v_cndmask_b32_e64 v34, 0, v1, s[4:5]
+; GFX9-NEXT: v_cndmask_b32_e64 v38, 0, v2, s[6:7]
+; GFX9-NEXT: v_cndmask_b32_e64 v36, 0, v3, s[8:9]
+; GFX9-NEXT: v_cndmask_b32_e64 v32, 0, v4, s[10:11]
+; GFX9-NEXT: v_cndmask_b32_e64 v35, 0, v5, s[12:13]
+; GFX9-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[14:15]
+; GFX9-NEXT: v_cndmask_b32_e64 v37, 0, v7, s[16:17]
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v8
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v9
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v10
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v11
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v12
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[28:29], -1, v13
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[40:41], -1, v14
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[42:43], -1, v15
; GFX9-NEXT: v_cndmask_b32_e32 v0, 0, v0, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v49, 0, v31, vcc
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v1
-; GFX9-NEXT: v_cndmask_b32_e32 v34, 0, v1, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v39, 0, v31, vcc
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v2
-; GFX9-NEXT: v_cndmask_b32_e32 v35, 0, v2, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v32, 0, v31, vcc
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v3
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v4
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v5
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v6
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v7
-; GFX9-NEXT: v_cndmask_b32_e32 v36, 0, v3, vcc
-; GFX9-NEXT: v_cndmask_b32_e64 v48, 0, v4, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e64 v37, 0, v5, s[6:7]
-; GFX9-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[8:9]
-; GFX9-NEXT: v_cndmask_b32_e64 v38, 0, v7, s[10:11]
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v8
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v9
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v10
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v11
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v12
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v13
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v14
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v15
-; GFX9-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[12:13]
-; GFX9-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[14:15]
-; GFX9-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[16:17]
-; GFX9-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[18:19]
-; GFX9-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[20:21]
-; GFX9-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[22:23]
-; GFX9-NEXT: v_cndmask_b32_e64 v28, 0, v14, s[24:25]
-; GFX9-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[26:27]
-; GFX9-NEXT: v_cndmask_b32_e32 v7, 0, v31, vcc
-; GFX9-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[6:7]
-; GFX9-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[8:9]
-; GFX9-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[10:11]
-; GFX9-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[12:13]
-; GFX9-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[14:15]
-; GFX9-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[16:17]
-; GFX9-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[18:19]
-; GFX9-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[20:21]
-; GFX9-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[22:23]
-; GFX9-NEXT: v_cndmask_b32_e64 v29, 0, v31, s[24:25]
-; GFX9-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[26:27]
-; GFX9-NEXT: v_mov_b32_e32 v1, v49
+; GFX9-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[18:19]
+; GFX9-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[20:21]
+; GFX9-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[22:23]
+; GFX9-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[24:25]
+; GFX9-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[26:27]
+; GFX9-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[28:29]
+; GFX9-NEXT: v_cndmask_b32_e64 v28, 0, v14, s[40:41]
+; GFX9-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[42:43]
+; GFX9-NEXT: v_cndmask_b32_e32 v1, 0, v31, vcc
+; GFX9-NEXT: v_cndmask_b32_e64 v3, 0, v31, s[4:5]
+; GFX9-NEXT: v_cndmask_b32_e64 v5, 0, v31, s[6:7]
+; GFX9-NEXT: v_cndmask_b32_e64 v7, 0, v31, s[8:9]
+; GFX9-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[10:11]
+; GFX9-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[12:13]
+; GFX9-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[14:15]
+; GFX9-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[16:17]
+; GFX9-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[18:19]
+; GFX9-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[20:21]
+; GFX9-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[22:23]
+; GFX9-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[24:25]
+; GFX9-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[26:27]
+; GFX9-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[28:29]
+; GFX9-NEXT: v_cndmask_b32_e64 v29, 0, v31, s[40:41]
+; GFX9-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[42:43]
; GFX9-NEXT: v_mov_b32_e32 v2, v34
-; GFX9-NEXT: v_mov_b32_e32 v3, v39
-; GFX9-NEXT: v_mov_b32_e32 v4, v35
-; GFX9-NEXT: v_mov_b32_e32 v5, v32
+; GFX9-NEXT: v_mov_b32_e32 v4, v38
; GFX9-NEXT: v_mov_b32_e32 v6, v36
-; GFX9-NEXT: v_mov_b32_e32 v8, v48
-; GFX9-NEXT: v_mov_b32_e32 v10, v37
+; GFX9-NEXT: v_mov_b32_e32 v8, v32
+; GFX9-NEXT: v_mov_b32_e32 v10, v35
; GFX9-NEXT: v_mov_b32_e32 v12, v33
-; GFX9-NEXT: v_mov_b32_e32 v14, v38
+; GFX9-NEXT: v_mov_b32_e32 v14, v37
; GFX9-NEXT: s_setpc_b64 s[30:31]
%cast = addrspacecast <16 x ptr addrspace(5)> %ptr to <16 x ptr>
ret <16 x ptr> %cast
@@ -1188,130 +1182,124 @@ define <16 x ptr> @addrspacecast_v16p3_to_v16p0(<16 x ptr addrspace(3)> %ptr) {
; CI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; CI-NEXT: s_load_dword s4, s[6:7], 0x10
; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v0
-; CI-NEXT: v_cndmask_b32_e32 v0, 0, v0, vcc
-; CI-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v5
-; CI-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v6
+; CI-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v2
+; CI-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v3
+; CI-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v4
; CI-NEXT: s_waitcnt lgkmcnt(0)
; CI-NEXT: v_mov_b32_e32 v31, s4
-; CI-NEXT: v_cndmask_b32_e32 v49, 0, v31, vcc
-; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v1
-; CI-NEXT: v_cndmask_b32_e32 v34, 0, v1, vcc
-; CI-NEXT: v_cndmask_b32_e32 v39, 0, v31, vcc
-; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v2
-; CI-NEXT: v_cndmask_b32_e32 v35, 0, v2, vcc
-; CI-NEXT: v_cndmask_b32_e32 v32, 0, v31, vcc
-; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v3
-; CI-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v4
-; CI-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v7
-; CI-NEXT: v_cndmask_b32_e32 v36, 0, v3, vcc
-; CI-NEXT: v_cndmask_b32_e64 v48, 0, v4, s[4:5]
-; CI-NEXT: v_cndmask_b32_e64 v37, 0, v5, s[6:7]
-; CI-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[8:9]
-; CI-NEXT: v_cndmask_b32_e64 v38, 0, v7, s[10:11]
-; CI-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v8
-; CI-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v9
-; CI-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v10
-; CI-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v11
-; CI-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v12
-; CI-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v13
-; CI-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v14
-; CI-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v15
-; CI-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[12:13]
-; CI-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[14:15]
-; CI-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[16:17]
-; CI-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[18:19]
-; CI-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[20:21]
-; CI-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[22:23]
-; CI-NEXT: v_cndmask_b32_e64 v28, 0, v14, s[24:25]
-; CI-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[26:27]
-; CI-NEXT: v_cndmask_b32_e32 v7, 0, v31, vcc
-; CI-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[4:5]
-; CI-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[6:7]
-; CI-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[8:9]
-; CI-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[10:11]
-; CI-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[12:13]
-; CI-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[14:15]
-; CI-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[16:17]
-; CI-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[18:19]
-; CI-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[20:21]
-; CI-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[22:23]
-; CI-NEXT: v_cndmask_b32_e64 v29, 0, v31, s[24:25]
-; CI-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[26:27]
-; CI-NEXT: v_mov_b32_e32 v1, v49
+; CI-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v1
+; CI-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v5
+; CI-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v6
+; CI-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v7
+; CI-NEXT: v_cndmask_b32_e32 v0, 0, v0, vcc
+; CI-NEXT: v_cndmask_b32_e64 v34, 0, v1, s[4:5]
+; CI-NEXT: v_cndmask_b32_e64 v38, 0, v2, s[6:7]
+; CI-NEXT: v_cndmask_b32_e64 v36, 0, v3, s[8:9]
+; CI-NEXT: v_cndmask_b32_e64 v32, 0, v4, s[10:11]
+; CI-NEXT: v_cndmask_b32_e64 v35, 0, v5, s[12:13]
+; CI-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[14:15]
+; CI-NEXT: v_cndmask_b32_e64 v37, 0, v7, s[16:17]
+; CI-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v8
+; CI-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v9
+; CI-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v10
+; CI-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v11
+; CI-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v12
+; CI-NEXT: v_cmp_ne_u32_e64 s[28:29], -1, v13
+; CI-NEXT: v_cndmask_b32_e32 v1, 0, v31, vcc
+; CI-NEXT: v_cmp_ne_u32_e32 vcc, -1, v14
+; CI-NEXT: v_cmp_ne_u32_e64 s[40:41], -1, v15
+; CI-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[18:19]
+; CI-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[20:21]
+; CI-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[22:23]
+; CI-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[24:25]
+; CI-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[26:27]
+; CI-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[28:29]
+; CI-NEXT: v_cndmask_b32_e32 v28, 0, v14, vcc
+; CI-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[40:41]
+; CI-NEXT: v_cndmask_b32_e64 v3, 0, v31, s[4:5]
+; CI-NEXT: v_cndmask_b32_e64 v5, 0, v31, s[6:7]
+; CI-NEXT: v_cndmask_b32_e64 v7, 0, v31, s[8:9]
+; CI-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[10:11]
+; CI-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[12:13]
+; CI-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[14:15]
+; CI-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[16:17]
+; CI-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[18:19]
+; CI-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[20:21]
+; CI-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[22:23]
+; CI-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[24:25]
+; CI-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[26:27]
+; CI-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[28:29]
+; CI-NEXT: v_cndmask_b32_e32 v29, 0, v31, vcc
+; CI-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[40:41]
; CI-NEXT: v_mov_b32_e32 v2, v34
-; CI-NEXT: v_mov_b32_e32 v3, v39
-; CI-NEXT: v_mov_b32_e32 v4, v35
-; CI-NEXT: v_mov_b32_e32 v5, v32
+; CI-NEXT: v_mov_b32_e32 v4, v38
; CI-NEXT: v_mov_b32_e32 v6, v36
-; CI-NEXT: v_mov_b32_e32 v8, v48
-; CI-NEXT: v_mov_b32_e32 v10, v37
+; CI-NEXT: v_mov_b32_e32 v8, v32
+; CI-NEXT: v_mov_b32_e32 v10, v35
; CI-NEXT: v_mov_b32_e32 v12, v33
-; CI-NEXT: v_mov_b32_e32 v14, v38
+; CI-NEXT: v_mov_b32_e32 v14, v37
; CI-NEXT: s_setpc_b64 s[30:31]
;
; GFX9-LABEL: addrspacecast_v16p3_to_v16p0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX9-NEXT: s_mov_b64 s[4:5], src_shared_base
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v0
; GFX9-NEXT: v_mov_b32_e32 v31, s5
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v1
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v2
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v3
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v4
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v5
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v6
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v7
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v0
+; GFX9-NEXT: v_cndmask_b32_e64 v34, 0, v1, s[4:5]
+; GFX9-NEXT: v_cndmask_b32_e64 v38, 0, v2, s[6:7]
+; GFX9-NEXT: v_cndmask_b32_e64 v36, 0, v3, s[8:9]
+; GFX9-NEXT: v_cndmask_b32_e64 v32, 0, v4, s[10:11]
+; GFX9-NEXT: v_cndmask_b32_e64 v35, 0, v5, s[12:13]
+; GFX9-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[14:15]
+; GFX9-NEXT: v_cndmask_b32_e64 v37, 0, v7, s[16:17]
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v8
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v9
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v10
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v11
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v12
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[28:29], -1, v13
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[40:41], -1, v14
+; GFX9-NEXT: v_cmp_ne_u32_e64 s[42:43], -1, v15
; GFX9-NEXT: v_cndmask_b32_e32 v0, 0, v0, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v49, 0, v31, vcc
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v1
-; GFX9-NEXT: v_cndmask_b32_e32 v34, 0, v1, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v39, 0, v31, vcc
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v2
-; GFX9-NEXT: v_cndmask_b32_e32 v35, 0, v2, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v32, 0, v31, vcc
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, -1, v3
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[4:5], -1, v4
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[6:7], -1, v5
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[8:9], -1, v6
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[10:11], -1, v7
-; GFX9-NEXT: v_cndmask_b32_e32 v36, 0, v3, vcc
-; GFX9-NEXT: v_cndmask_b32_e64 v48, 0, v4, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e64 v37, 0, v5, s[6:7]
-; GFX9-NEXT: v_cndmask_b32_e64 v33, 0, v6, s[8:9]
-; GFX9-NEXT: v_cndmask_b32_e64 v38, 0, v7, s[10:11]
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[12:13], -1, v8
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[14:15], -1, v9
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[16:17], -1, v10
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[18:19], -1, v11
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[20:21], -1, v12
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[22:23], -1, v13
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[24:25], -1, v14
-; GFX9-NEXT: v_cmp_ne_u32_e64 s[26:27], -1, v15
-; GFX9-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[12:13]
-; GFX9-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[14:15]
-; GFX9-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[16:17]
-; GFX9-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[18:19]
-; GFX9-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[20:21]
-; GFX9-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[22:23]
-; GFX9-NEXT: v_cndmask_b32_e64 v28, 0, v14, s[24:25]
-; GFX9-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[26:27]
-; GFX9-NEXT: v_cndmask_b32_e32 v7, 0, v31, vcc
-; GFX9-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[6:7]
-; GFX9-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[8:9]
-; GFX9-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[10:11]
-; GFX9-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[12:13]
-; GFX9-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[14:15]
-; GFX9-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[16:17]
-; GFX9-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[18:19]
-; GFX9-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[20:21]
-; GFX9-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[22:23]
-; GFX9-NEXT: v_cndmask_b32_e64 v29, 0, v31, s[24:25]
-; GFX9-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[26:27]
-; GFX9-NEXT: v_mov_b32_e32 v1, v49
+; GFX9-NEXT: v_cndmask_b32_e64 v16, 0, v8, s[18:19]
+; GFX9-NEXT: v_cndmask_b32_e64 v18, 0, v9, s[20:21]
+; GFX9-NEXT: v_cndmask_b32_e64 v20, 0, v10, s[22:23]
+; GFX9-NEXT: v_cndmask_b32_e64 v22, 0, v11, s[24:25]
+; GFX9-NEXT: v_cndmask_b32_e64 v24, 0, v12, s[26:27]
+; GFX9-NEXT: v_cndmask_b32_e64 v26, 0, v13, s[28:29]
+; GFX9-NEXT: v_cndmask_b32_e64 v28, 0, v14, s[40:41]
+; GFX9-NEXT: v_cndmask_b32_e64 v30, 0, v15, s[42:43]
+; GFX9-NEXT: v_cndmask_b32_e32 v1, 0, v31, vcc
+; GFX9-NEXT: v_cndmask_b32_e64 v3, 0, v31, s[4:5]
+; GFX9-NEXT: v_cndmask_b32_e64 v5, 0, v31, s[6:7]
+; GFX9-NEXT: v_cndmask_b32_e64 v7, 0, v31, s[8:9]
+; GFX9-NEXT: v_cndmask_b32_e64 v9, 0, v31, s[10:11]
+; GFX9-NEXT: v_cndmask_b32_e64 v11, 0, v31, s[12:13]
+; GFX9-NEXT: v_cndmask_b32_e64 v13, 0, v31, s[14:15]
+; GFX9-NEXT: v_cndmask_b32_e64 v15, 0, v31, s[16:17]
+; GFX9-NEXT: v_cndmask_b32_e64 v17, 0, v31, s[18:19]
+; GFX9-NEXT: v_cndmask_b32_e64 v19, 0, v31, s[20:21]
+; GFX9-NEXT: v_cndmask_b32_e64 v21, 0, v31, s[22:23]
+; GFX9-NEXT: v_cndmask_b32_e64 v23, 0, v31, s[24:25]
+; GFX9-NEXT: v_cndmask_b32_e64 v25, 0, v31, s[26:27]
+; GFX9-NEXT: v_cndmask_b32_e64 v27, 0, v31, s[28:29]
+; GFX9-NEXT: v_cndmask_b32_e64 v29, 0, v31, s[40:41]
+; GFX9-NEXT: v_cndmask_b32_e64 v31, 0, v31, s[42:43]
; GFX9-NEXT: v_mov_b32_e32 v2, v34
-; GFX9-NEXT: v_mov_b32_e32 v3, v39
-; GFX9-NEXT: v_mov_b32_e32 v4, v35
-; GFX9-NEXT: v_mov_b32_e32 v5, v32
+; GFX9-NEXT: v_mov_b32_e32 v4, v38
; GFX9-NEXT: v_mov_b32_e32 v6, v36
-; GFX9-NEXT: v_mov_b32_e32 v8, v48
-; GFX9-NEXT: v_mov_b32_e32 v10, v37
+; GFX9-NEXT: v_mov_b32_e32 v8, v32
+; GFX9-NEXT: v_mov_b32_e32 v10, v35
; GFX9-NEXT: v_mov_b32_e32 v12, v33
-; GFX9-NEXT: v_mov_b32_e32 v14, v38
+; GFX9-NEXT: v_mov_b32_e32 v14, v37
; GFX9-NEXT: s_setpc_b64 s[30:31]
%cast = addrspacecast <16 x ptr addrspace(3)> %ptr to <16 x ptr>
ret <16 x ptr> %cast
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
index 88ef88da73bce..4c6add4244f43 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
@@ -17751,341 +17751,366 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: v_readfirstlane_b32 s58, v10
-; SI-NEXT: v_readfirstlane_b32 s59, v9
-; SI-NEXT: v_readfirstlane_b32 s60, v8
-; SI-NEXT: v_readfirstlane_b32 s61, v7
-; SI-NEXT: v_readfirstlane_b32 s76, v0
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:324
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:316
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:300
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:292
-; SI-NEXT: v_readfirstlane_b32 s46, v30
-; SI-NEXT: v_readfirstlane_b32 s6, v29
-; SI-NEXT: v_readfirstlane_b32 s7, v28
-; SI-NEXT: v_readfirstlane_b32 s8, v27
-; SI-NEXT: v_readfirstlane_b32 s9, v26
-; SI-NEXT: v_readfirstlane_b32 s10, v25
-; SI-NEXT: v_readfirstlane_b32 s11, v24
-; SI-NEXT: v_readfirstlane_b32 s12, v23
-; SI-NEXT: v_readfirstlane_b32 s13, v22
-; SI-NEXT: v_readfirstlane_b32 s14, v21
-; SI-NEXT: v_readfirstlane_b32 s15, v20
-; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:16
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:24
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:32
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:144
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:152
+; SI-NEXT: v_readfirstlane_b32 s75, v0
; SI-NEXT: v_readfirstlane_b32 s41, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:332
+; SI-NEXT: v_readfirstlane_b32 s15, v20
; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:16
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:24
-; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:32
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:40
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:96
+; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: v_readfirstlane_b32 s7, v28
+; SI-NEXT: v_readfirstlane_b32 s9, v26
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:192
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:200
; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:208
; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:216
+; SI-NEXT: v_readfirstlane_b32 s76, v30
+; SI-NEXT: v_readfirstlane_b32 s6, v29
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:160
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:168
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:176
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s46, v13
+; SI-NEXT: v_readfirstlane_b32 s47, v12
+; SI-NEXT: v_readfirstlane_b32 s56, v11
+; SI-NEXT: v_readfirstlane_b32 s45, v14
+; SI-NEXT: v_readfirstlane_b32 s43, v16
+; SI-NEXT: v_readfirstlane_b32 s44, v15
+; SI-NEXT: v_readfirstlane_b32 s74, v1
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:60
; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:224
; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:232
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:240
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:144
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:152
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:176
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:184
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:192
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:200
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:208
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:216
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:224
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:232
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:240
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:248
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:256
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:264
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:272
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:280
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:288
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:296
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:304
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:312
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:320
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:328
-; SI-NEXT: v_readfirstlane_b32 s43, v16
-; SI-NEXT: v_readfirstlane_b32 s44, v15
-; SI-NEXT: v_readfirstlane_b32 s45, v14
-; SI-NEXT: v_readfirstlane_b32 s47, v13
-; SI-NEXT: v_readfirstlane_b32 s56, v12
-; SI-NEXT: v_readfirstlane_b32 s57, v11
-; SI-NEXT: v_readfirstlane_b32 s62, v6
-; SI-NEXT: v_readfirstlane_b32 s63, v5
-; SI-NEXT: v_readfirstlane_b32 s72, v4
-; SI-NEXT: v_readfirstlane_b32 s73, v3
-; SI-NEXT: v_readfirstlane_b32 s74, v2
-; SI-NEXT: v_readfirstlane_b32 s75, v1
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:248
+; SI-NEXT: v_readfirstlane_b32 s8, v27
+; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: v_readfirstlane_b32 s11, v24
+; SI-NEXT: v_readfirstlane_b32 s12, v23
+; SI-NEXT: v_readfirstlane_b32 s13, v22
+; SI-NEXT: v_readfirstlane_b32 s14, v21
+; SI-NEXT: v_readfirstlane_b32 s57, v10
+; SI-NEXT: v_readfirstlane_b32 s58, v9
+; SI-NEXT: v_readfirstlane_b32 s59, v8
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_readfirstlane_b32 s62, v5
+; SI-NEXT: v_readfirstlane_b32 s72, v3
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:172
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:180
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:212
+; SI-NEXT: v_readfirstlane_b32 s60, v7
+; SI-NEXT: v_readfirstlane_b32 s63, v4
+; SI-NEXT: v_readfirstlane_b32 s73, v2
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:148
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v16, 8, v33
-; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v34
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v42
-; SI-NEXT: v_lshlrev_b32_e32 v14, 24, v43
-; SI-NEXT: v_lshlrev_b32_e32 v51, 24, v45
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v47
-; SI-NEXT: v_lshlrev_b32_e32 v33, 24, v59
-; SI-NEXT: v_lshlrev_b32_e32 v42, 24, v61
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v20
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v21
-; SI-NEXT: v_lshlrev_b32_e32 v22, 24, v22
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v17
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
+; SI-NEXT: s_and_b64 s[4:5], vcc, exec
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
+; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v37
+; SI-NEXT: v_lshlrev_b32_e32 v50, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v49
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v39
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v38
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v36
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v46
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v33
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v56
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v58
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:84
+; SI-NEXT: v_lshlrev_b32_e32 v45, 8, v34
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v48
+; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 24, v28
+; SI-NEXT: v_lshlrev_b32_e32 v24, 8, v57
+; SI-NEXT: v_lshlrev_b32_e32 v9, 24, v58
+; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v30
+; SI-NEXT: v_lshlrev_b32_e32 v33, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v30, 24, v56
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v59
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v60
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v61
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v62
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v60
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v53
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v62
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:92
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:100
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:108
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v55
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v53
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(14) expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v52
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:132
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v30
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v43
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v27
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v46
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:156
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v49
-; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v18
-; SI-NEXT: v_lshlrev_b32_e32 v17, 8, v36
-; SI-NEXT: v_lshlrev_b32_e32 v18, 24, v41
-; SI-NEXT: v_lshlrev_b32_e32 v44, 24, v57
-; SI-NEXT: v_lshlrev_b32_e32 v55, 24, v24
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v25
-; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:264
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:272
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:280
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:252
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:260
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:276
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v46, 8, v39
-; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v37
-; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v35
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v32
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:284
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:276
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:268
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:260
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:252
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:244
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:228
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:220
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:180
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:172
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:164
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:156
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:220
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:228
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:236
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:148
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:140
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:132
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:20
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:12
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:4
-; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_lshlrev_b32_e32 v31, 8, v31
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:244
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v11
+; SI-NEXT: s_waitcnt vmcnt(11)
+; SI-NEXT: v_lshlrev_b32_e32 v32, 24, v12
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v13
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:288
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:296
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:304
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:312
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:284
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:292
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:308
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v29, 24, v29
+; SI-NEXT: s_waitcnt vmcnt(7)
+; SI-NEXT: v_lshlrev_b32_e32 v12, 8, v12
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v36
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshlrev_b32_e32 v42, 8, v38
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v54
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:320
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:324
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v54
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v46
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:12
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; SI-NEXT: s_cbranch_scc0 .LBB15_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_mov_b32_e32 v40, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; SI-NEXT: s_and_b32 s4, s46, 0xff
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v6, v15, v6
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_or_b32_e32 v38, v38, v11
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v17
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v16
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v38
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v33, v4
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_or_b32_e32 v2, v42, v2
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v35
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_or_b32_e32 v60, v37, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v59
+; SI-NEXT: v_or_b32_e32 v35, v35, v50
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v50, 0xffff, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v58
+; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v35
+; SI-NEXT: v_or_b32_e32 v48, v48, v58
+; SI-NEXT: v_mov_b32_e32 v58, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_and_b32 s4, s76, 0xff
+; SI-NEXT: v_or_b32_e32 v51, s4, v51
+; SI-NEXT: v_and_b32_e32 v51, 0xffff, v51
+; SI-NEXT: v_mov_b32_e32 v62, v32
+; SI-NEXT: v_mov_b32_e32 v32, v13
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v57
+; SI-NEXT: v_mov_b32_e32 v35, v11
+; SI-NEXT: v_or_b32_e32 v11, v51, v60
+; SI-NEXT: v_mov_b32_e32 v59, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v56
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_or_b32_e32 v12, v50, v48
+; SI-NEXT: v_mov_b32_e32 v37, v42
+; SI-NEXT: v_mov_b32_e32 v61, v40
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_and_b32_e32 v18, 0xff, v18
+; SI-NEXT: v_and_b32_e32 v19, 0xff, v19
+; SI-NEXT: v_and_b32_e32 v20, 0xff, v20
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: v_and_b32_e32 v23, 0xff, v23
+; SI-NEXT: v_or_b32_e32 v1, v1, v4
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v20, v0
-; SI-NEXT: s_lshl_b32 s5, s17, 8
-; SI-NEXT: s_lshl_b32 s77, s19, 24
-; SI-NEXT: s_lshl_b32 s78, s23, 24
-; SI-NEXT: s_lshl_b32 s79, s27, 24
-; SI-NEXT: s_lshl_b32 s88, s75, 24
-; SI-NEXT: s_lshl_b32 s89, s63, 24
-; SI-NEXT: s_lshl_b32 s90, s59, 24
-; SI-NEXT: s_lshl_b32 s91, s47, 24
-; SI-NEXT: s_lshl_b32 s92, s42, 24
-; SI-NEXT: s_lshl_b32 s93, s14, 24
-; SI-NEXT: s_lshl_b32 s94, s10, 24
-; SI-NEXT: s_lshl_b32 s95, s6, 24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v7, s4, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_or_b32_e32 v11, v7, v6
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v63
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v7, v18, v7
-; SI-NEXT: v_mov_b32_e32 v6, v13
-; SI-NEXT: v_or_b32_e32 v13, v5, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v62
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v60
-; SI-NEXT: v_or_b32_e32 v5, v5, v19
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v14, v7
-; SI-NEXT: v_mov_b32_e32 v60, v14
-; SI-NEXT: v_or_b32_e32 v14, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v57
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v0, v2, v0
+; SI-NEXT: v_and_b32_e32 v27, 0xff, v27
+; SI-NEXT: v_and_b32_e32 v22, 0xff, v22
+; SI-NEXT: v_or_b32_e32 v23, v26, v23
+; SI-NEXT: v_or_b32_e32 v26, v1, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v55
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v61
+; SI-NEXT: v_or_b32_e32 v27, v27, v58
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
+; SI-NEXT: v_or_b32_e32 v22, v30, v22
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v62, v1
+; SI-NEXT: v_or_b32_e32 v22, v27, v22
+; SI-NEXT: v_or_b32_e32 v27, v0, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v59
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v29, v1
+; SI-NEXT: v_or_b32_e32 v28, v0, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
+; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s5, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s77, s19, 24
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s5, s77, s5
; SI-NEXT: s_or_b32 s4, s4, s5
@@ -18094,6 +18119,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s5, s5, s77
; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s77, s78, s77
; SI-NEXT: s_or_b32 s5, s5, s77
@@ -18102,38 +18128,48 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s26, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
+; SI-NEXT: s_lshl_b32 s79, s27, 24
; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_or_b32 s78, s79, s78
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s28, 0xff
; SI-NEXT: s_lshl_b32 s79, s29, 8
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_mov_b32_e32 v51, v33
+; SI-NEXT: v_or_b32_e32 v13, v13, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s76, 0xff
+; SI-NEXT: s_and_b32 s79, s75, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s78, s78, 0xffff
; SI-NEXT: s_or_b32 s79, s88, s79
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s74, 0xff
-; SI-NEXT: s_lshl_b32 s88, s73, 8
+; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_lshl_b32 s88, s72, 8
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s72, 0xff
+; SI-NEXT: s_and_b32 s88, s63, 0xff
; SI-NEXT: s_lshl_b32 s88, s88, 16
+; SI-NEXT: s_lshl_b32 s89, s62, 24
; SI-NEXT: s_and_b32 s79, s79, 0xffff
; SI-NEXT: s_or_b32 s88, s89, s88
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s62, 0xff
-; SI-NEXT: s_lshl_b32 s89, s61, 8
+; SI-NEXT: s_and_b32 s88, s61, 0xff
+; SI-NEXT: s_lshl_b32 s89, s60, 8
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s60, 0xff
+; SI-NEXT: s_and_b32 s89, s59, 0xff
; SI-NEXT: s_lshl_b32 s89, s89, 16
+; SI-NEXT: s_lshl_b32 s90, s58, 24
; SI-NEXT: s_and_b32 s88, s88, 0xffff
; SI-NEXT: s_or_b32 s89, s90, s89
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s58, 0xff
-; SI-NEXT: s_lshl_b32 s90, s57, 8
+; SI-NEXT: s_and_b32 s89, s57, 0xff
+; SI-NEXT: s_lshl_b32 s90, s56, 8
; SI-NEXT: s_or_b32 s89, s89, s90
-; SI-NEXT: s_and_b32 s90, s56, 0xff
+; SI-NEXT: s_and_b32 s90, s47, 0xff
; SI-NEXT: s_lshl_b32 s90, s90, 16
+; SI-NEXT: s_lshl_b32 s91, s46, 24
; SI-NEXT: s_and_b32 s89, s89, 0xffff
; SI-NEXT: s_or_b32 s90, s91, s90
; SI-NEXT: s_or_b32 s89, s89, s90
@@ -18142,6 +18178,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s90, s90, s91
; SI-NEXT: s_and_b32 s91, s43, 0xff
; SI-NEXT: s_lshl_b32 s91, s91, 16
+; SI-NEXT: s_lshl_b32 s92, s42, 24
; SI-NEXT: s_and_b32 s90, s90, 0xffff
; SI-NEXT: s_or_b32 s91, s92, s91
; SI-NEXT: s_or_b32 s90, s90, s91
@@ -18150,6 +18187,8 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s91, s91, s92
; SI-NEXT: s_and_b32 s92, s15, 0xff
; SI-NEXT: s_lshl_b32 s92, s92, 16
+; SI-NEXT: s_lshl_b32 s93, s14, 24
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
; SI-NEXT: s_and_b32 s91, s91, 0xffff
; SI-NEXT: s_or_b32 s92, s93, s92
; SI-NEXT: s_or_b32 s91, s91, s92
@@ -18158,180 +18197,178 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s11, 0xff
; SI-NEXT: s_lshl_b32 s93, s93, 16
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v51, v7
-; SI-NEXT: v_or_b32_e32 v15, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v32
+; SI-NEXT: s_lshl_b32 s94, s10, 24
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: s_and_b32 s92, s92, 0xffff
; SI-NEXT: s_or_b32 s93, s94, s93
+; SI-NEXT: v_or_b32_e32 v8, v8, v24
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s9, 0xff
; SI-NEXT: s_lshl_b32 s94, s8, 8
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
+; SI-NEXT: v_or_b32_e32 v5, v9, v5
; SI-NEXT: s_or_b32 s93, s93, s94
; SI-NEXT: s_and_b32 s94, s7, 0xff
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v21, 0xff, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xff, v25
+; SI-NEXT: v_or_b32_e32 v24, v8, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v6
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_mov_b32_e32 v50, v33
+; SI-NEXT: v_or_b32_e32 v14, v33, v14
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v13, v13, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v47
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: s_lshl_b32 s94, s94, 16
+; SI-NEXT: s_lshl_b32 s95, s6, 24
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_or_b32_e32 v25, v25, v39
+; SI-NEXT: v_or_b32_e32 v5, v5, v10
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: s_and_b32 s93, s93, 0xffff
; SI-NEXT: s_or_b32 s94, s95, s94
-; SI-NEXT: s_or_b32 s93, s93, s94
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v45
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v48, v7
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_or_b32_e32 v16, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v43
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v52
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v44, v7
-; SI-NEXT: v_or_b32_e32 v17, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v41
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v21, v63, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v25
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v18, v5, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v5, s88
-; SI-NEXT: v_mov_b32_e32 v7, s90
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v19, v3, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v3, v7, v3
+; SI-NEXT: s_or_b32 s93, s93, s94
+; SI-NEXT: v_or_b32_e32 v23, v25, v23
+; SI-NEXT: v_or_b32_e32 v25, v5, v3
+; SI-NEXT: v_mov_b32_e32 v2, s77
; SI-NEXT: v_mov_b32_e32 v3, s78
; SI-NEXT: v_mov_b32_e32 v4, s79
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v20, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v54
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v5, s88
+; SI-NEXT: v_mov_b32_e32 v6, s89
+; SI-NEXT: v_mov_b32_e32 v7, s90
+; SI-NEXT: v_mov_b32_e32 v8, s91
+; SI-NEXT: v_mov_b32_e32 v9, s92
+; SI-NEXT: v_mov_b32_e32 v10, s93
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v21, v1
-; SI-NEXT: v_or_b32_e32 v21, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v50
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v49
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v22, v1
-; SI-NEXT: v_or_b32_e32 v22, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v39
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v23
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v23, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v27
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v48, v33
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v46
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
+; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_or_b32_e32 v33, v42, v33
+; SI-NEXT: v_mov_b32_e32 v46, v44
+; SI-NEXT: v_mov_b32_e32 v44, v42
+; SI-NEXT: v_mov_b32_e32 v42, v38
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_mov_b32_e32 v38, v40
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v40
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v33, v33, v45
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v24
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v55, v1
-; SI-NEXT: v_or_b32_e32 v24, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v59, v1
-; SI-NEXT: v_or_b32_e32 v25, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v29
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v26
-; SI-NEXT: v_or_b32_e32 v0, v0, v31
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v56, v1
-; SI-NEXT: v_or_b32_e32 v26, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v30
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_and_b32_e32 v1, 0xff, v35
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v27, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v61
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v28, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v6
-; SI-NEXT: v_mov_b32_e32 v6, s89
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_or_b32_e32 v1, v43, v1
; SI-NEXT: v_or_b32_e32 v29, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v10
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v0, v0, v46
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v32
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v34
+; SI-NEXT: v_or_b32_e32 v0, v0, v37
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v36, v1
; SI-NEXT: v_or_b32_e32 v30, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v8
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v40
-; SI-NEXT: v_or_b32_e32 v0, v0, v34
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v42
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v52
+; SI-NEXT: v_or_b32_e32 v0, v0, v46
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v37, v1
+; SI-NEXT: v_or_b32_e32 v1, v54, v1
; SI-NEXT: v_or_b32_e32 v31, v0, v1
; SI-NEXT: v_mov_b32_e32 v0, s4
; SI-NEXT: v_mov_b32_e32 v1, s5
-; SI-NEXT: v_mov_b32_e32 v2, s77
-; SI-NEXT: v_mov_b32_e32 v8, s91
-; SI-NEXT: v_mov_b32_e32 v9, s92
-; SI-NEXT: v_mov_b32_e32 v10, s93
; SI-NEXT: s_mov_b64 s[4:5], 0
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v15, v15, v40
+; SI-NEXT: v_or_b32_e32 v15, v33, v15
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v16, v40
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v33
+; SI-NEXT: v_or_b32_e32 v33, v33, v41
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v17, v17, v40
+; SI-NEXT: v_or_b32_e32 v17, v33, v17
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v18
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v18, v18, v40
+; SI-NEXT: v_or_b32_e32 v18, v33, v18
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v19, v19, v40
+; SI-NEXT: v_or_b32_e32 v19, v33, v19
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v20, v20, v40
+; SI-NEXT: v_or_b32_e32 v20, v33, v20
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: v_or_b32_e32 v21, v33, v21
; SI-NEXT: s_branch .LBB15_3
; SI-NEXT: .LBB15_2:
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
-; SI-NEXT: v_mov_b32_e32 v60, v14
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15_vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23_vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31
; SI-NEXT: .LBB15_3: ; %Flow
@@ -18340,9 +18377,9 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB15_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
-; SI-NEXT: s_add_i32 s46, s46, 3
-; SI-NEXT: s_and_b32 s4, s46, 0xff
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; SI-NEXT: s_add_i32 s76, s76, 3
+; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
@@ -18353,14 +18390,14 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s76, s76, 3
-; SI-NEXT: s_add_i32 s74, s74, 3
-; SI-NEXT: s_add_i32 s72, s72, 3
-; SI-NEXT: s_add_i32 s62, s62, 3
-; SI-NEXT: s_add_i32 s60, s60, 3
-; SI-NEXT: s_add_i32 s58, s58, 3
-; SI-NEXT: s_add_i32 s56, s56, 3
-; SI-NEXT: s_and_b32 s22, s56, 0xff
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s63, s63, 3
+; SI-NEXT: s_add_i32 s61, s61, 3
+; SI-NEXT: s_add_i32 s59, s59, 3
+; SI-NEXT: s_add_i32 s57, s57, 3
+; SI-NEXT: s_add_i32 s47, s47, 3
+; SI-NEXT: s_and_b32 s22, s47, 0xff
; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_add_i32 s45, s45, 3
; SI-NEXT: s_add_i32 s43, s43, 3
@@ -18398,22 +18435,22 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
; SI-NEXT: v_mov_b32_e32 v9, s10
; SI-NEXT: v_mov_b32_e32 v10, s6
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, s4, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -18445,38 +18482,38 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s17, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s29, 8
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s19, s76, 0xff
+; SI-NEXT: s_and_b32 s19, s75, 0xff
; SI-NEXT: s_addk_i32 s17, 0x300
-; SI-NEXT: s_lshl_b32 s18, s75, 24
+; SI-NEXT: s_lshl_b32 s18, s74, 24
; SI-NEXT: s_lshl_b32 s19, s19, 16
; SI-NEXT: s_and_b32 s17, s17, 0xffff
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s18, s74, 0xff
-; SI-NEXT: s_lshl_b32 s19, s73, 8
+; SI-NEXT: s_and_b32 s18, s73, 0xff
+; SI-NEXT: s_lshl_b32 s19, s72, 8
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_and_b32 s20, s63, 0xff
; SI-NEXT: s_addk_i32 s18, 0x300
-; SI-NEXT: s_lshl_b32 s19, s63, 24
+; SI-NEXT: s_lshl_b32 s19, s62, 24
; SI-NEXT: s_lshl_b32 s20, s20, 16
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s19, s62, 0xff
-; SI-NEXT: s_lshl_b32 s20, s61, 8
+; SI-NEXT: s_and_b32 s19, s61, 0xff
+; SI-NEXT: s_lshl_b32 s20, s60, 8
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s21, s60, 0xff
+; SI-NEXT: s_and_b32 s21, s59, 0xff
; SI-NEXT: s_addk_i32 s19, 0x300
-; SI-NEXT: s_lshl_b32 s20, s59, 24
+; SI-NEXT: s_lshl_b32 s20, s58, 24
; SI-NEXT: s_lshl_b32 s21, s21, 16
; SI-NEXT: s_and_b32 s19, s19, 0xffff
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s20, s58, 0xff
-; SI-NEXT: s_lshl_b32 s21, s57, 8
+; SI-NEXT: s_and_b32 s20, s57, 0xff
+; SI-NEXT: s_lshl_b32 s21, s56, 8
; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_addk_i32 s20, 0x300
-; SI-NEXT: s_lshl_b32 s21, s47, 24
+; SI-NEXT: s_lshl_b32 s21, s46, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_or_b32 s20, s21, s20
@@ -18496,7 +18533,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_or_b32_e32 v2, v2, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x300, v2
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: s_or_b32 s22, s23, s22
@@ -18524,333 +18561,330 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_or_b32_e32 v0, v3, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; SI-NEXT: v_mov_b32_e32 v3, s17
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v38, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v51, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v63, v0
+; SI-NEXT: v_or_b32_e32 v0, v50, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v62, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v48, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v60, v0
+; SI-NEXT: v_or_b32_e32 v0, v44, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
+; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v38
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v16, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v17, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v18, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v20, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v21, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v22, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v24, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v25, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v26, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v27, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v28, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v29, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -18860,21 +18894,21 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v30, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -18899,7 +18933,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0)
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
;
; VI-LABEL: bitcast_v128i8_to_v32i32_scalar:
@@ -18924,8 +18958,9 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; VI-NEXT: v_readfirstlane_b32 s73, v3
+; VI-NEXT: v_readfirstlane_b32 s74, v2
; VI-NEXT: v_readfirstlane_b32 s47, v13
+; VI-NEXT: v_readfirstlane_b32 s72, v4
; VI-NEXT: v_readfirstlane_b32 s61, v7
; VI-NEXT: v_readfirstlane_b32 s62, v6
; VI-NEXT: v_readfirstlane_b32 s63, v5
@@ -18951,101 +18986,100 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
-; VI-NEXT: v_readfirstlane_b32 s72, v4
-; VI-NEXT: v_readfirstlane_b32 s74, v2
+; VI-NEXT: v_readfirstlane_b32 s73, v3
; VI-NEXT: v_readfirstlane_b32 s75, v1
-; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
+; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:316
+; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
+; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:308
; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:296
; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; VI-NEXT: s_waitcnt vmcnt(13)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: s_waitcnt vmcnt(10)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:320
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:268
+; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:264
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:248
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
-; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:220
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; VI-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; VI-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; VI-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; VI-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:180
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:148
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:136
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -19066,186 +19100,192 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:48
; VI-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; VI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: v_mov_b32_e32 v20, 0xc0c0004
-; VI-NEXT: v_mov_b32_e32 v4, s23
-; VI-NEXT: v_mov_b32_e32 v2, s19
-; VI-NEXT: v_mov_b32_e32 v3, s21
-; VI-NEXT: v_perm_b32 v4, s22, v4, v20
+; VI-NEXT: v_mov_b32_e32 v40, 0xc0c0004
+; VI-NEXT: v_mov_b32_e32 v3, s19
; VI-NEXT: v_mov_b32_e32 v13, s17
-; VI-NEXT: v_perm_b32 v2, s18, v2, v20
-; VI-NEXT: v_perm_b32 v3, s20, v3, v20
-; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_perm_b32 v13, s16, v13, v20
+; VI-NEXT: v_perm_b32 v3, s18, v3, v40
+; VI-NEXT: v_mov_b32_e32 v2, s23
+; VI-NEXT: v_perm_b32 v13, s16, v13, v40
+; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; VI-NEXT: v_mov_b32_e32 v42, s21
+; VI-NEXT: v_perm_b32 v2, s22, v2, v40
+; VI-NEXT: v_perm_b32 v42, s20, v42, v40
; VI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; VI-NEXT: v_mov_b32_e32 v19, v1
-; VI-NEXT: v_or_b32_e32 v1, v3, v4
+; VI-NEXT: v_mov_b32_e32 v20, v0
+; VI-NEXT: v_or_b32_e32 v0, v13, v3
; VI-NEXT: v_mov_b32_e32 v3, s27
-; VI-NEXT: v_mov_b32_e32 v22, v0
-; VI-NEXT: v_or_b32_e32 v0, v13, v2
+; VI-NEXT: v_mov_b32_e32 v19, v1
+; VI-NEXT: v_or_b32_e32 v1, v42, v2
; VI-NEXT: v_mov_b32_e32 v2, s25
-; VI-NEXT: v_perm_b32 v3, s26, v3, v20
-; VI-NEXT: v_perm_b32 v2, s24, v2, v20
+; VI-NEXT: v_perm_b32 v3, s26, v3, v40
+; VI-NEXT: v_mov_b32_e32 v59, v4
+; VI-NEXT: v_perm_b32 v2, s24, v2, v40
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; VI-NEXT: v_mov_b32_e32 v4, s75
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s76, v4, v20
-; VI-NEXT: v_mov_b32_e32 v44, v5
-; VI-NEXT: v_perm_b32 v3, s28, v3, v20
+; VI-NEXT: v_perm_b32 v4, s76, v4, v40
+; VI-NEXT: v_mov_b32_e32 v47, v5
+; VI-NEXT: v_perm_b32 v3, s28, v3, v40
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; VI-NEXT: v_mov_b32_e32 v5, s63
; VI-NEXT: v_or_b32_e32 v3, v3, v4
; VI-NEXT: v_mov_b32_e32 v4, s73
-; VI-NEXT: v_perm_b32 v5, s72, v5, v20
-; VI-NEXT: v_mov_b32_e32 v46, v6
-; VI-NEXT: v_perm_b32 v4, s74, v4, v20
+; VI-NEXT: v_perm_b32 v5, s72, v5, v40
+; VI-NEXT: v_mov_b32_e32 v57, v6
+; VI-NEXT: v_perm_b32 v4, s74, v4, v40
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; VI-NEXT: v_mov_b32_e32 v6, s59
; VI-NEXT: v_or_b32_e32 v4, v4, v5
; VI-NEXT: v_mov_b32_e32 v5, s61
-; VI-NEXT: v_perm_b32 v6, s60, v6, v20
-; VI-NEXT: v_mov_b32_e32 v45, v7
-; VI-NEXT: v_perm_b32 v5, s62, v5, v20
+; VI-NEXT: v_perm_b32 v6, s60, v6, v40
+; VI-NEXT: v_mov_b32_e32 v56, v7
+; VI-NEXT: v_perm_b32 v5, s62, v5, v40
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; VI-NEXT: v_mov_b32_e32 v7, s47
; VI-NEXT: v_or_b32_e32 v5, v5, v6
; VI-NEXT: v_mov_b32_e32 v6, s57
-; VI-NEXT: v_perm_b32 v7, s56, v7, v20
-; VI-NEXT: v_mov_b32_e32 v47, v40
-; VI-NEXT: v_mov_b32_e32 v40, v8
-; VI-NEXT: v_perm_b32 v6, s58, v6, v20
+; VI-NEXT: v_perm_b32 v7, s56, v7, v40
+; VI-NEXT: v_mov_b32_e32 v43, v8
+; VI-NEXT: v_perm_b32 v6, s58, v6, v40
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_mov_b32_e32 v7, s45
-; VI-NEXT: v_perm_b32 v8, s44, v8, v20
-; VI-NEXT: v_mov_b32_e32 v43, v9
-; VI-NEXT: v_perm_b32 v7, s46, v7, v20
+; VI-NEXT: v_perm_b32 v8, s44, v8, v40
+; VI-NEXT: v_mov_b32_e32 v46, v9
+; VI-NEXT: v_perm_b32 v7, s46, v7, v40
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s15
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s41
-; VI-NEXT: v_perm_b32 v9, s40, v9, v20
-; VI-NEXT: v_mov_b32_e32 v56, v41
-; VI-NEXT: v_mov_b32_e32 v41, v10
-; VI-NEXT: v_perm_b32 v8, s42, v8, v20
+; VI-NEXT: v_perm_b32 v9, s40, v9, v40
+; VI-NEXT: v_mov_b32_e32 v44, v10
+; VI-NEXT: v_perm_b32 v8, s42, v8, v40
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_or_b32_e32 v8, v8, v9
; VI-NEXT: v_mov_b32_e32 v9, s13
-; VI-NEXT: v_perm_b32 v10, s12, v10, v20
-; VI-NEXT: v_mov_b32_e32 v42, v11
-; VI-NEXT: v_perm_b32 v9, s14, v9, v20
+; VI-NEXT: v_perm_b32 v10, s12, v10, v40
+; VI-NEXT: v_mov_b32_e32 v58, v41
+; VI-NEXT: v_mov_b32_e32 v41, v11
+; VI-NEXT: v_perm_b32 v9, s14, v9, v40
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_or_b32_e32 v9, v9, v10
; VI-NEXT: v_mov_b32_e32 v10, s9
-; VI-NEXT: v_perm_b32 v11, s8, v11, v20
-; VI-NEXT: v_perm_b32 v10, s10, v10, v20
+; VI-NEXT: v_perm_b32 v11, s8, v11, v40
+; VI-NEXT: v_perm_b32 v10, s10, v10, v40
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
+; VI-NEXT: v_perm_b32 v18, v41, v18, s4
+; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; VI-NEXT: v_perm_b32 v19, v63, v19, s4
+; VI-NEXT: v_or_b32_e32 v18, v19, v18
+; VI-NEXT: v_perm_b32 v19, v22, v45, s4
+; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; VI-NEXT: v_perm_b32 v20, v62, v20, s4
+; VI-NEXT: v_or_b32_e32 v19, v20, v19
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: v_perm_b32 v12, v12, v61, s4
; VI-NEXT: v_perm_b32 v14, v52, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_perm_b32 v15, v48, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_perm_b32 v16, v50, v16, s4
-; VI-NEXT: v_perm_b32 v18, v42, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_perm_b32 v17, v35, v17, s4
-; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
-; VI-NEXT: v_perm_b32 v19, v63, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
-; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_perm_b32 v19, v23, v24, s4
-; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; VI-NEXT: v_perm_b32 v23, v34, v33, s4
-; VI-NEXT: v_perm_b32 v24, v41, v43, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v11, s6, v11, v20
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v22, v30, v31, s4
+; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: v_perm_b32 v11, s6, v11, v40
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: s_waitcnt vmcnt(2)
+; VI-NEXT: v_perm_b32 v20, v25, v20, s4
+; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v25, v56, v57, s4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v20, v13, s4
+; VI-NEXT: v_perm_b32 v13, v40, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_e32 v11, v11, v13
; VI-NEXT: v_perm_b32 v13, v21, v60, s4
@@ -19253,88 +19293,84 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v27, v29, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; VI-NEXT: v_perm_b32 v20, v57, v28, s4
-; VI-NEXT: v_or_b32_e32 v13, v20, v13
-; VI-NEXT: v_perm_b32 v20, v55, v51, s4
-; VI-NEXT: v_or_b32_e32 v14, v20, v14
-; VI-NEXT: v_perm_b32 v20, v54, v53, s4
-; VI-NEXT: v_or_b32_e32 v15, v20, v15
-; VI-NEXT: v_perm_b32 v20, v39, v36, s4
-; VI-NEXT: v_or_b32_e32 v16, v20, v16
-; VI-NEXT: v_perm_b32 v20, v49, v38, s4
-; VI-NEXT: v_or_b32_e32 v17, v20, v17
-; VI-NEXT: v_perm_b32 v20, v62, v22, s4
-; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
-; VI-NEXT: v_perm_b32 v21, v25, v58, s4
-; VI-NEXT: v_perm_b32 v22, v30, v31, s4
-; VI-NEXT: v_perm_b32 v25, v45, v46, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v26, v20, s4
-; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v21, v26, v28, s4
+; VI-NEXT: v_or_b32_e32 v13, v21, v13
+; VI-NEXT: v_perm_b32 v21, v55, v32, s4
+; VI-NEXT: v_or_b32_e32 v14, v21, v14
+; VI-NEXT: v_perm_b32 v21, v54, v53, s4
+; VI-NEXT: v_or_b32_e32 v15, v21, v15
+; VI-NEXT: v_perm_b32 v21, v39, v36, s4
+; VI-NEXT: v_or_b32_e32 v16, v21, v16
+; VI-NEXT: v_perm_b32 v21, v49, v38, s4
+; VI-NEXT: v_or_b32_e32 v17, v21, v17
+; VI-NEXT: v_perm_b32 v21, v23, v24, s4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v23, v34, v33, s4
+; VI-NEXT: v_perm_b32 v24, v44, v46, s4
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v37, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v32, v22, s4
+; VI-NEXT: v_perm_b32 v22, v51, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v40, v23, s4
+; VI-NEXT: v_perm_b32 v23, v43, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v44, v24, s4
+; VI-NEXT: v_perm_b32 v24, v47, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v25, v26, s4
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v25
+; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v25
; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v26, v25, s4
-; VI-NEXT: v_or_b32_e32 v25, v25, v27
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v25, v59, v25, s4
+; VI-NEXT: v_or_b32_e32 v25, v25, v26
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v26, v26, v27, s4
+; VI-NEXT: v_perm_b32 v26, v26, v58, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_or_b32_e32 v26, v26, v28
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v27, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; VI-NEXT: v_or_b32_e32 v27, v27, v29
-; VI-NEXT: v_perm_b32 v29, v47, v56, s4
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v28, v59, v28, s4
+; VI-NEXT: v_perm_b32 v28, v29, v28, s4
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v28, v29, v28
; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
-; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v30, v31, v30, s4
@@ -19349,16 +19385,16 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v31, v32, v31, s4
; VI-NEXT: v_or_b32_e32 v30, v31, v30
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v31, v32, v31, s4
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v31, v31, v32, s4
+; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v31
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v32, v33, v32, s4
-; VI-NEXT: v_or_b32_e32 v31, v32, v31
+; VI-NEXT: v_perm_b32 v31, v33, v31, s4
+; VI-NEXT: v_or_b32_e32 v31, v31, v32
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
@@ -19470,7 +19506,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x300, v10
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_add_u32_e32 v0, vcc, 0x3000000, v0
@@ -19485,12 +19521,12 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v9, vcc, 0x3000000, v9
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x3000000, v10
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v12, s6, v11, v12
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v12
+; VI-NEXT: v_perm_b32 v11, s6, v11, v12
+; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v11
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v11, v13, v11, s4
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
@@ -19501,57 +19537,57 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v12
; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v13, v12, s4
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
+; VI-NEXT: v_perm_b32 v12, v13, v12, s4
+; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v12
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v12, v14, v12, s4
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v12, v12, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v12, vcc, 0x3000000, v12
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v14, v14, v13, s4
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
+; VI-NEXT: v_perm_b32 v13, v14, v13, s4
+; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v13
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v15, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x3000000, v13
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v15, v15, v14, s4
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v15
+; VI-NEXT: v_perm_b32 v14, v15, v14, s4
+; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v14
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v14, v16, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v14, v14, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x3000000, v14
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v15
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v16, v16, v15, s4
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v16
+; VI-NEXT: v_perm_b32 v15, v16, v15, s4
+; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v15
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v15
; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
@@ -19559,151 +19595,151 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_perm_b32 v15, v17, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_or_b32_sdwa v15, v15, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v15, vcc, 0x3000000, v15
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v17, v17, v16, s4
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v17
+; VI-NEXT: v_perm_b32 v16, v17, v16, s4
+; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v16
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v16, v18, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_or_b32_sdwa v16, v16, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v16, vcc, 0x3000000, v16
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v18, v18, v17, s4
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v18
+; VI-NEXT: v_perm_b32 v17, v18, v17, s4
+; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v17
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v17, v19, v17, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; VI-NEXT: v_or_b32_sdwa v17, v17, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v17, vcc, 0x3000000, v17
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v19, v19, v18, s4
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v19
+; VI-NEXT: v_perm_b32 v18, v19, v18, s4
+; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v18
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v18, v20, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; VI-NEXT: v_or_b32_sdwa v18, v18, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v18, vcc, 0x3000000, v18
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v20, v19, s4
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v20
+; VI-NEXT: v_perm_b32 v19, v20, v19, s4
+; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v19
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v19, v21, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; VI-NEXT: v_or_b32_sdwa v19, v19, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v19, vcc, 0x3000000, v19
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v21, v21, v20, s4
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v21
+; VI-NEXT: v_perm_b32 v20, v21, v20, s4
+; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v20
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v20, v22, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; VI-NEXT: v_or_b32_sdwa v20, v20, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v20, vcc, 0x3000000, v20
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v22, v21, s4
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v22
+; VI-NEXT: v_perm_b32 v21, v22, v21, s4
+; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v21
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v23, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_sdwa v21, v21, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v21, vcc, 0x3000000, v21
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v23, v22, s4
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v23
+; VI-NEXT: v_perm_b32 v22, v23, v22, s4
+; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v22
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v22, v24, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_sdwa v22, v22, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v22, vcc, 0x3000000, v22
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v24, v23, s4
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v24
+; VI-NEXT: v_perm_b32 v23, v24, v23, s4
+; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v23
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v23, v25, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_sdwa v23, v23, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v23, vcc, 0x3000000, v23
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v25, v24, s4
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v25
+; VI-NEXT: v_perm_b32 v24, v25, v24, s4
+; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v24
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v24, v26, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_sdwa v24, v24, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v24, vcc, 0x3000000, v24
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v25
@@ -19711,55 +19747,55 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v26, v25, s4
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x300, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v27, v25, s4
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_or_b32_sdwa v25, v25, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v25, vcc, 0x3000000, v25
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x300, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; VI-NEXT: v_or_b32_sdwa v26, v26, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x3000000, v26
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x300, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v29, v27, s4
; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; VI-NEXT: v_or_b32_sdwa v27, v27, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x3000000, v27
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v28, v29, v28, s4
; VI-NEXT: v_add_u32_e32 v29, vcc, 0x300, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v28
; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
@@ -19767,11 +19803,11 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_perm_b32 v28, v30, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; VI-NEXT: v_or_b32_sdwa v28, v28, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x3000000, v28
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v29
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_add_u32_e32 v30, vcc, 0x300, v29
@@ -19858,8 +19894,9 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; GFX9-NEXT: v_readfirstlane_b32 s73, v3
-; GFX9-NEXT: v_readfirstlane_b32 s47, v13
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s72, v4
+; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s61, v7
; GFX9-NEXT: v_readfirstlane_b32 s62, v6
; GFX9-NEXT: v_readfirstlane_b32 s63, v5
@@ -19876,31 +19913,28 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s13, v23
; GFX9-NEXT: v_readfirstlane_b32 s14, v22
; GFX9-NEXT: v_readfirstlane_b32 s15, v21
-; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s41, v19
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
+; GFX9-NEXT: v_readfirstlane_b32 s47, v13
; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
-; GFX9-NEXT: v_readfirstlane_b32 s72, v4
-; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s73, v3
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
-; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
+; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:320
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:316
+; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:296
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; GFX9-NEXT: s_waitcnt vmcnt(13)
+; GFX9-NEXT: s_waitcnt vmcnt(11)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -19910,82 +19944,86 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; GFX9-NEXT: s_nop 0
+; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:264
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:248
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
+; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; GFX9-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; GFX9-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; GFX9-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
-; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
+; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:176
; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; GFX9-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:148
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:136
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -19993,108 +20031,109 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:108
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:104
-; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:100
; GFX9-NEXT: buffer_load_ushort v38, off, s[0:3], s32 offset:96
-; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v50, off, s[0:3], s32 offset:80
; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v15, off, s[0:3], s32 offset:72
-; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:68
-; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:64
-; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
-; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
+; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:68
+; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:64
+; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:60
+; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:56
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(55)
-; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(54)
-; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(49)
-; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(62)
; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
@@ -20114,7 +20153,8 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_mov_b32_e32 v19, v1
; GFX9-NEXT: v_or_b32_e32 v1, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v3, s27
-; GFX9-NEXT: v_mov_b32_e32 v22, v0
+; GFX9-NEXT: v_mov_b32_e32 v46, v42
+; GFX9-NEXT: v_mov_b32_e32 v42, v0
; GFX9-NEXT: v_or_b32_e32 v0, v13, v2
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v20
@@ -20124,28 +20164,28 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_or_b32_e32 v2, v2, v3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v20
-; GFX9-NEXT: v_mov_b32_e32 v44, v5
+; GFX9-NEXT: v_mov_b32_e32 v45, v5
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s63
; GFX9-NEXT: v_or_b32_e32 v3, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v4, s73
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v20
-; GFX9-NEXT: v_mov_b32_e32 v46, v6
+; GFX9-NEXT: v_mov_b32_e32 v56, v6
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s59
; GFX9-NEXT: v_or_b32_e32 v4, v4, v5
; GFX9-NEXT: v_mov_b32_e32 v5, s61
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v20
-; GFX9-NEXT: v_mov_b32_e32 v45, v7
+; GFX9-NEXT: v_mov_b32_e32 v47, v7
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s47
; GFX9-NEXT: v_or_b32_e32 v5, v5, v6
; GFX9-NEXT: v_mov_b32_e32 v6, s57
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v20
-; GFX9-NEXT: v_mov_b32_e32 v47, v40
+; GFX9-NEXT: v_mov_b32_e32 v57, v40
; GFX9-NEXT: v_mov_b32_e32 v40, v8
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
@@ -20153,22 +20193,21 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_or_b32_e32 v6, v6, v7
; GFX9-NEXT: v_mov_b32_e32 v7, s45
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v20
-; GFX9-NEXT: v_mov_b32_e32 v43, v9
+; GFX9-NEXT: v_mov_b32_e32 v44, v9
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v9, s15
; GFX9-NEXT: v_or_b32_e32 v7, v7, v8
; GFX9-NEXT: v_mov_b32_e32 v8, s41
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v20
-; GFX9-NEXT: v_mov_b32_e32 v56, v41
-; GFX9-NEXT: v_mov_b32_e32 v41, v10
+; GFX9-NEXT: v_mov_b32_e32 v43, v10
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; GFX9-NEXT: v_mov_b32_e32 v10, s11
; GFX9-NEXT: v_or_b32_e32 v8, v8, v9
; GFX9-NEXT: v_mov_b32_e32 v9, s13
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v20
-; GFX9-NEXT: v_mov_b32_e32 v42, v11
+; GFX9-NEXT: v_mov_b32_e32 v25, v11
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: v_mov_b32_e32 v11, s7
@@ -20181,22 +20220,21 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v12, v12, v61, s4
-; GFX9-NEXT: v_perm_b32 v14, v52, v14, s4
-; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v32, v15, s4
+; GFX9-NEXT: v_perm_b32 v14, v33, v14, s4
+; GFX9-NEXT: v_perm_b32 v15, v39, v15, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; GFX9-NEXT: v_perm_b32 v16, v53, v16, s4
-; GFX9-NEXT: v_perm_b32 v18, v42, v18, s4
+; GFX9-NEXT: v_perm_b32 v18, v25, v18, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; GFX9-NEXT: v_perm_b32 v17, v35, v17, s4
+; GFX9-NEXT: v_perm_b32 v17, v49, v17, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; GFX9-NEXT: v_perm_b32 v19, v63, v19, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; GFX9-NEXT: v_or_b32_e32 v18, v19, v18
-; GFX9-NEXT: v_perm_b32 v19, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v19, v22, v41, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; GFX9-NEXT: v_perm_b32 v23, v34, v33, s4
-; GFX9-NEXT: v_perm_b32 v24, v41, v43, s4
+; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
+; GFX9-NEXT: v_perm_b32 v25, v47, v56, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v11, s6, v11, v20
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
@@ -20210,81 +20248,83 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_or_b32_e32 v12, v12, v13
; GFX9-NEXT: v_perm_b32 v13, v27, v29, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; GFX9-NEXT: v_perm_b32 v20, v57, v28, s4
+; GFX9-NEXT: v_perm_b32 v20, v26, v28, s4
; GFX9-NEXT: v_or_b32_e32 v13, v20, v13
-; GFX9-NEXT: v_perm_b32 v20, v55, v48, s4
-; GFX9-NEXT: v_or_b32_e32 v14, v20, v14
-; GFX9-NEXT: v_perm_b32 v20, v54, v39, s4
+; GFX9-NEXT: v_perm_b32 v20, v52, v48, s4
+; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; GFX9-NEXT: v_or_b32_e32 v14, v14, v20
+; GFX9-NEXT: v_perm_b32 v20, v32, v54, s4
; GFX9-NEXT: v_or_b32_e32 v15, v20, v15
; GFX9-NEXT: v_perm_b32 v20, v36, v50, s4
; GFX9-NEXT: v_or_b32_e32 v16, v20, v16
-; GFX9-NEXT: v_perm_b32 v20, v49, v38, s4
+; GFX9-NEXT: v_perm_b32 v20, v35, v38, s4
; GFX9-NEXT: v_or_b32_e32 v17, v20, v17
-; GFX9-NEXT: v_perm_b32 v20, v62, v22, s4
+; GFX9-NEXT: v_perm_b32 v20, v62, v42, s4
; GFX9-NEXT: v_or_b32_e32 v19, v20, v19
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v21, v25, v58, s4
-; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
-; GFX9-NEXT: v_perm_b32 v25, v45, v46, s4
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v21, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v23, v34, v55, s4
+; GFX9-NEXT: v_perm_b32 v24, v43, v44, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v26, v20, s4
+; GFX9-NEXT: v_perm_b32 v20, v58, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; GFX9-NEXT: v_or_b32_e32 v20, v21, v20
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v21, v37, v21, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; GFX9-NEXT: v_or_b32_e32 v21, v22, v21
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v22, v51, v22, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; GFX9-NEXT: v_or_b32_e32 v22, v23, v22
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v23, v40, v23, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; GFX9-NEXT: v_or_b32_e32 v23, v24, v23
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v44, v24, s4
+; GFX9-NEXT: v_perm_b32 v24, v45, v24, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; GFX9-NEXT: v_or_b32_e32 v24, v25, v24
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v26, s4
+; GFX9-NEXT: v_perm_b32 v25, v25, v46, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_or_b32_e32 v25, v25, v27
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v26, v27, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_or_b32_e32 v26, v26, v28
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v27, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: v_or_b32_e32 v27, v27, v29
-; GFX9-NEXT: v_perm_b32 v29, v47, v56, s4
-; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
; GFX9-NEXT: v_perm_b32 v28, v59, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_perm_b32 v29, v57, v29, s4
; GFX9-NEXT: v_or_b32_e32 v28, v29, v28
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
@@ -20306,10 +20346,10 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_or_b32_e32 v30, v31, v30
-; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
+; GFX9-NEXT: v_perm_b32 v31, v31, v32, s4
; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -20332,21 +20372,21 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_mov_b32_e32 v12, 0xc0c0004
; GFX9-NEXT: s_add_i32 s18, s18, 3
; GFX9-NEXT: v_mov_b32_e32 v1, s19
-; GFX9-NEXT: s_add_i32 s20, s20, 3
-; GFX9-NEXT: v_mov_b32_e32 v2, s21
-; GFX9-NEXT: s_add_i32 s22, s22, 3
-; GFX9-NEXT: v_mov_b32_e32 v3, s23
; GFX9-NEXT: v_perm_b32 v0, s16, v0, v12
-; GFX9-NEXT: s_movk_i32 s4, 0x300
+; GFX9-NEXT: s_movk_i32 s5, 0x300
; GFX9-NEXT: v_perm_b32 v1, s18, v1, v12
-; GFX9-NEXT: v_perm_b32 v2, s20, v2, v12
-; GFX9-NEXT: v_perm_b32 v3, s22, v3, v12
; GFX9-NEXT: v_add_u32_e32 v0, 0x300, v0
-; GFX9-NEXT: v_add_u32_sdwa v1, v1, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v1, v1, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_or_b32_sdwa v1, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX9-NEXT: s_add_i32 s20, s20, 3
+; GFX9-NEXT: v_mov_b32_e32 v1, s21
+; GFX9-NEXT: s_add_i32 s22, s22, 3
+; GFX9-NEXT: v_mov_b32_e32 v2, s23
+; GFX9-NEXT: v_perm_b32 v1, s20, v1, v12
+; GFX9-NEXT: v_perm_b32 v2, s22, v2, v12
+; GFX9-NEXT: v_add_u32_e32 v1, 0x300, v1
+; GFX9-NEXT: v_add_u32_sdwa v2, v2, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s24, s24, 3
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: s_add_i32 s26, s26, 3
@@ -20354,7 +20394,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v12
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v12
; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v3, v3, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v2, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s28, s28, 3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
@@ -20363,7 +20403,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v12
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v12
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
-; GFX9-NEXT: v_add_u32_sdwa v4, v4, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v3, v3, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_mov_b32_e32 v4, s73
@@ -20372,7 +20412,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v12
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v12
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
-; GFX9-NEXT: v_add_u32_sdwa v5, v5, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v4, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s62, s62, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s61
@@ -20381,7 +20421,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v12
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v12
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u32_sdwa v6, v6, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v5, v5, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s58, s58, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s57
@@ -20390,7 +20430,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v12
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v12
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
-; GFX9-NEXT: v_add_u32_sdwa v7, v7, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v6, v6, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s46, s46, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s45
@@ -20399,7 +20439,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v12
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v12
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
-; GFX9-NEXT: v_add_u32_sdwa v8, v8, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v8, v8, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v7, v7, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v8, s41
@@ -20408,7 +20448,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v12
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v12
; GFX9-NEXT: v_add_u32_e32 v8, 0x300, v8
-; GFX9-NEXT: v_add_u32_sdwa v9, v9, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v9, v9, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v8, v8, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s14, s14, 3
; GFX9-NEXT: v_mov_b32_e32 v9, s13
@@ -20417,7 +20457,7 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v12
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v12
; GFX9-NEXT: v_add_u32_e32 v9, 0x300, v9
-; GFX9-NEXT: v_add_u32_sdwa v10, v10, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v10, v10, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v9, v9, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s10, s10, 3
; GFX9-NEXT: v_mov_b32_e32 v10, s9
@@ -20426,321 +20466,321 @@ define inreg <32 x i32> @bitcast_v128i8_to_v32i32_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v10, s10, v10, v12
; GFX9-NEXT: v_perm_b32 v11, s8, v11, v12
; GFX9-NEXT: v_add_u32_e32 v10, 0x300, v10
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_add_i32 s6, s6, 3
-; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
+; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, s6, v11, v12
+; GFX9-NEXT: v_perm_b32 v11, s6, v11, v12
+; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v12
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v11, v13, v11, s5
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v11, v13, v11, s4
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v12
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v13, v12, s5
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
+; GFX9-NEXT: v_perm_b32 v12, v13, v12, s4
+; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v12
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, v14, v12, s5
-; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v12, v14, v12, s4
+; GFX9-NEXT: v_add_u32_sdwa v12, v12, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v12, v13, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v14, v13, s5
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
+; GFX9-NEXT: v_perm_b32 v13, v14, v13, s4
+; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v13
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v15, v13, s5
-; GFX9-NEXT: v_add_u32_sdwa v13, v13, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v13, v15, v13, s4
+; GFX9-NEXT: v_add_u32_sdwa v13, v13, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v15, v14, s5
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v15
+; GFX9-NEXT: v_perm_b32 v14, v15, v14, s4
+; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v14
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v16, v14, s5
-; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v16, v14, s4
+; GFX9-NEXT: v_add_u32_sdwa v14, v14, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v16, v15, s5
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v16
+; GFX9-NEXT: v_perm_b32 v15, v16, v15, s4
+; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v15
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v17, v15, s5
-; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v15, v17, v15, s4
+; GFX9-NEXT: v_add_u32_sdwa v15, v15, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v15, v16, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v17, v16, s5
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v17
+; GFX9-NEXT: v_perm_b32 v16, v17, v16, s4
+; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v16
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v18, v16, s5
-; GFX9-NEXT: v_add_u32_sdwa v16, v16, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v16, v18, v16, s4
+; GFX9-NEXT: v_add_u32_sdwa v16, v16, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v16, v17, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v18, v17, s5
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v18
+; GFX9-NEXT: v_perm_b32 v17, v18, v17, s4
+; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v17
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v19, v17, s5
-; GFX9-NEXT: v_add_u32_sdwa v17, v17, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v17, v19, v17, s4
+; GFX9-NEXT: v_add_u32_sdwa v17, v17, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v17, v18, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v19, v18, s5
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v19
+; GFX9-NEXT: v_perm_b32 v18, v19, v18, s4
+; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v18
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v20, v18, s5
-; GFX9-NEXT: v_add_u32_sdwa v18, v18, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v18, v20, v18, s4
+; GFX9-NEXT: v_add_u32_sdwa v18, v18, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v18, v19, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v20, v19, s5
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v20
+; GFX9-NEXT: v_perm_b32 v19, v20, v19, s4
+; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v19
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v21, v19, s5
-; GFX9-NEXT: v_add_u32_sdwa v19, v19, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v19, v21, v19, s4
+; GFX9-NEXT: v_add_u32_sdwa v19, v19, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v19, v20, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v21, v20, s5
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v21
+; GFX9-NEXT: v_perm_b32 v20, v21, v20, s4
+; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v20
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v22, v20, s5
-; GFX9-NEXT: v_add_u32_sdwa v20, v20, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v20, v22, v20, s4
+; GFX9-NEXT: v_add_u32_sdwa v20, v20, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v20, v21, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v22, v21, s5
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v22
+; GFX9-NEXT: v_perm_b32 v21, v22, v21, s4
+; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v21
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v23, v21, s5
-; GFX9-NEXT: v_add_u32_sdwa v21, v21, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v21, v23, v21, s4
+; GFX9-NEXT: v_add_u32_sdwa v21, v21, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v21, v22, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v23, v22, s5
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v23
+; GFX9-NEXT: v_perm_b32 v22, v23, v22, s4
+; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v22
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v24, v22, s5
-; GFX9-NEXT: v_add_u32_sdwa v22, v22, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v22, v24, v22, s4
+; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v22, v23, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v24, v23, s5
+; GFX9-NEXT: v_perm_b32 v23, v24, v23, s4
+; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v23
; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v24
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v25, v23, s5
-; GFX9-NEXT: v_add_u32_sdwa v23, v23, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v23, v25, v23, s4
+; GFX9-NEXT: v_add_u32_sdwa v23, v23, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v23, v24, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v24, s5
+; GFX9-NEXT: v_perm_b32 v24, v25, v24, s4
+; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v24
; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v25
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v26, v24, s5
-; GFX9-NEXT: v_add_u32_sdwa v24, v24, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v24, v26, v24, s4
+; GFX9-NEXT: v_add_u32_sdwa v24, v24, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v24, v25, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v26, v25, s5
+; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_add_u32_e32 v26, 0x300, v25
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v27, v25, s5
-; GFX9-NEXT: v_add_u32_sdwa v25, v25, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v25, v27, v25, s4
+; GFX9-NEXT: v_add_u32_sdwa v25, v25, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v25, v26, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v27, v26, s5
+; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_add_u32_e32 v27, 0x300, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v28, v26, s5
-; GFX9-NEXT: v_add_u32_sdwa v26, v26, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v26, v28, v26, s4
+; GFX9-NEXT: v_add_u32_sdwa v26, v26, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v26, v27, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v28, v27, s5
+; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: v_add_u32_e32 v28, 0x300, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v29, v27, s5
-; GFX9-NEXT: v_add_u32_sdwa v27, v27, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v27, v29, v27, s4
+; GFX9-NEXT: v_add_u32_sdwa v27, v27, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v27, v28, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v28
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v29, v28, s5
+; GFX9-NEXT: v_perm_b32 v28, v29, v28, s4
; GFX9-NEXT: v_add_u32_e32 v29, 0x300, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v30, v28, s5
-; GFX9-NEXT: v_add_u32_sdwa v28, v28, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v28, v30, v28, s4
+; GFX9-NEXT: v_add_u32_sdwa v28, v28, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v28, v29, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v30, v29, s5
+; GFX9-NEXT: v_perm_b32 v29, v30, v29, s4
; GFX9-NEXT: v_add_u32_e32 v30, 0x300, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v31, v29, s5
-; GFX9-NEXT: v_add_u32_sdwa v29, v29, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v29, v31, v29, s4
+; GFX9-NEXT: v_add_u32_sdwa v29, v29, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v29, v30, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v31, v30, s5
+; GFX9-NEXT: v_perm_b32 v30, v31, v30, s4
; GFX9-NEXT: v_add_u32_e32 v31, 0x300, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v32, v30, s5
-; GFX9-NEXT: v_add_u32_sdwa v30, v30, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v30, v32, v30, s4
+; GFX9-NEXT: v_add_u32_sdwa v30, v30, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v30, v31, v30 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s5
+; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_add_u32_e32 v32, 0x300, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v33, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v33, v31, s5
-; GFX9-NEXT: v_add_u32_sdwa v31, v31, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v31, v33, v31, s4
+; GFX9-NEXT: v_add_u32_sdwa v31, v31, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v31, v32, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: .LBB15_5: ; %end
; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -31548,15 +31588,15 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v26
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v25
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v24
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v23
@@ -31569,12 +31609,12 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v21
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v20
; SI-NEXT: v_mov_b32_e32 v52, v18
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v19
; SI-NEXT: v_mov_b32_e32 v53, v17
@@ -31614,18 +31654,18 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v42
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v43
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v44
+; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v43
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v44
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v45
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v46
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v47
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v56
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v57
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v59
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v46
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v47
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v56
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v57
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v58
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v59
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v60
; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v61
-; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v62
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v62
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
@@ -31650,7 +31690,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v62
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v63
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v36
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v61
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v50
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -31659,31 +31699,31 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v32
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v59
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v49
; SI-NEXT: v_or_b32_e32 v3, v3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v35
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v48
; SI-NEXT: v_or_b32_e32 v5, v5, v6
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v56
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v39
; SI-NEXT: v_or_b32_e32 v6, v6, v7
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v33
; SI-NEXT: v_or_b32_e32 v7, v7, v8
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v46
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v38
; SI-NEXT: v_or_b32_e32 v8, v8, v9
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v45
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v34
; SI-NEXT: v_or_b32_e32 v9, v9, v10
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v44
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v37
; SI-NEXT: v_or_b32_e32 v10, v10, v11
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v43
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v63
; SI-NEXT: v_or_b32_e32 v11, v11, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v42
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v51
@@ -31741,25 +31781,25 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr42
-; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; kill: killed $vgpr51
; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
@@ -31787,7 +31827,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_or_b32_e32 v18, v18, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: s_waitcnt vmcnt(11)
@@ -31821,7 +31861,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_or_b32_e32 v19, v19, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v20, 0xffff, v20
; SI-NEXT: v_or_b32_e32 v20, v20, v21
@@ -31897,7 +31937,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_cbranch_execz .LBB22_4
; SI-NEXT: ; %bb.3: ; %cmp.true
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, v63
+; SI-NEXT: v_cvt_f32_f16_e32 v0, v36
; SI-NEXT: v_cvt_f32_f16_e32 v2, v50
; SI-NEXT: v_cvt_f32_f16_e32 v1, v62
; SI-NEXT: v_cvt_f32_f16_e32 v3, v61
@@ -31925,7 +31965,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v3, v36
+; SI-NEXT: v_cvt_f32_f16_e32 v3, v49
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_cvt_f32_f16_e32 v6, v57
@@ -31936,7 +31976,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, v49
+; SI-NEXT: v_cvt_f32_f16_e32 v4, v35
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_cvt_f32_f16_e32 v8, v47
@@ -31947,7 +31987,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v5, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v5, v33
+; SI-NEXT: v_cvt_f32_f16_e32 v5, v48
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_cvt_f32_f16_e32 v10, v45
@@ -31958,7 +31998,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, v48
+; SI-NEXT: v_cvt_f32_f16_e32 v6, v39
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_cvt_f32_f16_e32 v12, v43
@@ -31969,7 +32009,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v7, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v7, v35
+; SI-NEXT: v_cvt_f32_f16_e32 v7, v33
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f32_f16_e32 v14, v41
@@ -31980,7 +32020,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v8, v39
+; SI-NEXT: v_cvt_f32_f16_e32 v8, v38
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f32_f16_e32 v16, v55
@@ -32002,23 +32042,23 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v10, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v10, v38
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: v_cvt_f32_f16_e32 v10, v37
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v11, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v11, v37
+; SI-NEXT: v_cvt_f32_f16_e32 v11, v63
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -32147,7 +32187,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_cvt_f32_f16_e32 v25, v25
; SI-NEXT: s_waitcnt vmcnt(2)
@@ -32163,7 +32203,7 @@ define <32 x i32> @bitcast_v64f16_to_v32i32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -53865,341 +53905,366 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: v_readfirstlane_b32 s58, v10
-; SI-NEXT: v_readfirstlane_b32 s59, v9
-; SI-NEXT: v_readfirstlane_b32 s60, v8
-; SI-NEXT: v_readfirstlane_b32 s61, v7
-; SI-NEXT: v_readfirstlane_b32 s76, v0
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:324
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:316
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:300
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:292
-; SI-NEXT: v_readfirstlane_b32 s46, v30
-; SI-NEXT: v_readfirstlane_b32 s6, v29
-; SI-NEXT: v_readfirstlane_b32 s7, v28
-; SI-NEXT: v_readfirstlane_b32 s8, v27
-; SI-NEXT: v_readfirstlane_b32 s9, v26
-; SI-NEXT: v_readfirstlane_b32 s10, v25
-; SI-NEXT: v_readfirstlane_b32 s11, v24
-; SI-NEXT: v_readfirstlane_b32 s12, v23
-; SI-NEXT: v_readfirstlane_b32 s13, v22
-; SI-NEXT: v_readfirstlane_b32 s14, v21
-; SI-NEXT: v_readfirstlane_b32 s15, v20
-; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:16
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:24
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:32
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:144
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:152
+; SI-NEXT: v_readfirstlane_b32 s75, v0
; SI-NEXT: v_readfirstlane_b32 s41, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:332
+; SI-NEXT: v_readfirstlane_b32 s15, v20
; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:16
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:24
-; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:32
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:40
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:96
+; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: v_readfirstlane_b32 s7, v28
+; SI-NEXT: v_readfirstlane_b32 s9, v26
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:192
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:200
; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:208
; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:216
+; SI-NEXT: v_readfirstlane_b32 s76, v30
+; SI-NEXT: v_readfirstlane_b32 s6, v29
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:160
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:168
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:176
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s46, v13
+; SI-NEXT: v_readfirstlane_b32 s47, v12
+; SI-NEXT: v_readfirstlane_b32 s56, v11
+; SI-NEXT: v_readfirstlane_b32 s45, v14
+; SI-NEXT: v_readfirstlane_b32 s43, v16
+; SI-NEXT: v_readfirstlane_b32 s44, v15
+; SI-NEXT: v_readfirstlane_b32 s74, v1
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:60
; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:224
; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:232
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:240
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:144
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:152
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:176
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:184
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:192
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:200
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:208
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:216
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:224
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:232
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:240
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:248
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:256
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:264
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:272
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:280
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:288
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:296
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:304
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:312
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:320
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:328
-; SI-NEXT: v_readfirstlane_b32 s43, v16
-; SI-NEXT: v_readfirstlane_b32 s44, v15
-; SI-NEXT: v_readfirstlane_b32 s45, v14
-; SI-NEXT: v_readfirstlane_b32 s47, v13
-; SI-NEXT: v_readfirstlane_b32 s56, v12
-; SI-NEXT: v_readfirstlane_b32 s57, v11
-; SI-NEXT: v_readfirstlane_b32 s62, v6
-; SI-NEXT: v_readfirstlane_b32 s63, v5
-; SI-NEXT: v_readfirstlane_b32 s72, v4
-; SI-NEXT: v_readfirstlane_b32 s73, v3
-; SI-NEXT: v_readfirstlane_b32 s74, v2
-; SI-NEXT: v_readfirstlane_b32 s75, v1
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:248
+; SI-NEXT: v_readfirstlane_b32 s8, v27
+; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: v_readfirstlane_b32 s11, v24
+; SI-NEXT: v_readfirstlane_b32 s12, v23
+; SI-NEXT: v_readfirstlane_b32 s13, v22
+; SI-NEXT: v_readfirstlane_b32 s14, v21
+; SI-NEXT: v_readfirstlane_b32 s57, v10
+; SI-NEXT: v_readfirstlane_b32 s58, v9
+; SI-NEXT: v_readfirstlane_b32 s59, v8
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_readfirstlane_b32 s62, v5
+; SI-NEXT: v_readfirstlane_b32 s72, v3
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:172
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:180
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:212
+; SI-NEXT: v_readfirstlane_b32 s60, v7
+; SI-NEXT: v_readfirstlane_b32 s63, v4
+; SI-NEXT: v_readfirstlane_b32 s73, v2
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:148
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v16, 8, v33
-; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v34
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v42
-; SI-NEXT: v_lshlrev_b32_e32 v14, 24, v43
-; SI-NEXT: v_lshlrev_b32_e32 v51, 24, v45
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v47
-; SI-NEXT: v_lshlrev_b32_e32 v33, 24, v59
-; SI-NEXT: v_lshlrev_b32_e32 v42, 24, v61
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v20
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v21
-; SI-NEXT: v_lshlrev_b32_e32 v22, 24, v22
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v17
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
+; SI-NEXT: s_and_b64 s[4:5], vcc, exec
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
+; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v37
+; SI-NEXT: v_lshlrev_b32_e32 v50, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v49
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v39
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v38
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v36
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v46
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v33
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v56
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v58
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:84
+; SI-NEXT: v_lshlrev_b32_e32 v45, 8, v34
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v48
+; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 24, v28
+; SI-NEXT: v_lshlrev_b32_e32 v24, 8, v57
+; SI-NEXT: v_lshlrev_b32_e32 v9, 24, v58
+; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v30
+; SI-NEXT: v_lshlrev_b32_e32 v33, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v30, 24, v56
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v59
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v60
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v61
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v62
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v60
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v53
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v62
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:92
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:100
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:108
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v55
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v53
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(14) expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v52
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:132
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v30
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v43
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v27
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v46
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:156
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v49
-; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v18
-; SI-NEXT: v_lshlrev_b32_e32 v17, 8, v36
-; SI-NEXT: v_lshlrev_b32_e32 v18, 24, v41
-; SI-NEXT: v_lshlrev_b32_e32 v44, 24, v57
-; SI-NEXT: v_lshlrev_b32_e32 v55, 24, v24
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v25
-; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:264
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:272
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:280
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:252
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:260
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:276
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v46, 8, v39
-; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v37
-; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v35
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v32
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:284
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:276
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:268
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:260
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:252
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:244
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:228
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:220
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:180
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:172
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:164
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:156
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:220
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:228
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:236
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:148
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:140
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:132
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:20
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:12
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:4
-; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_lshlrev_b32_e32 v31, 8, v31
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:244
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v11
+; SI-NEXT: s_waitcnt vmcnt(11)
+; SI-NEXT: v_lshlrev_b32_e32 v32, 24, v12
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v13
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:288
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:296
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:304
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:312
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:284
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:292
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:308
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v29, 24, v29
+; SI-NEXT: s_waitcnt vmcnt(7)
+; SI-NEXT: v_lshlrev_b32_e32 v12, 8, v12
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v36
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshlrev_b32_e32 v42, 8, v38
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v54
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:320
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:324
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v54
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v46
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:12
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; SI-NEXT: s_cbranch_scc0 .LBB39_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_mov_b32_e32 v40, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; SI-NEXT: s_and_b32 s4, s46, 0xff
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v6, v15, v6
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_or_b32_e32 v38, v38, v11
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v17
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v16
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v38
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v33, v4
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_or_b32_e32 v2, v42, v2
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v35
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_or_b32_e32 v60, v37, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v59
+; SI-NEXT: v_or_b32_e32 v35, v35, v50
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v50, 0xffff, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v58
+; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v35
+; SI-NEXT: v_or_b32_e32 v48, v48, v58
+; SI-NEXT: v_mov_b32_e32 v58, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_and_b32 s4, s76, 0xff
+; SI-NEXT: v_or_b32_e32 v51, s4, v51
+; SI-NEXT: v_and_b32_e32 v51, 0xffff, v51
+; SI-NEXT: v_mov_b32_e32 v62, v32
+; SI-NEXT: v_mov_b32_e32 v32, v13
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v57
+; SI-NEXT: v_mov_b32_e32 v35, v11
+; SI-NEXT: v_or_b32_e32 v11, v51, v60
+; SI-NEXT: v_mov_b32_e32 v59, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v56
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_or_b32_e32 v12, v50, v48
+; SI-NEXT: v_mov_b32_e32 v37, v42
+; SI-NEXT: v_mov_b32_e32 v61, v40
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_and_b32_e32 v18, 0xff, v18
+; SI-NEXT: v_and_b32_e32 v19, 0xff, v19
+; SI-NEXT: v_and_b32_e32 v20, 0xff, v20
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: v_and_b32_e32 v23, 0xff, v23
+; SI-NEXT: v_or_b32_e32 v1, v1, v4
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v20, v0
-; SI-NEXT: s_lshl_b32 s5, s17, 8
-; SI-NEXT: s_lshl_b32 s77, s19, 24
-; SI-NEXT: s_lshl_b32 s78, s23, 24
-; SI-NEXT: s_lshl_b32 s79, s27, 24
-; SI-NEXT: s_lshl_b32 s88, s75, 24
-; SI-NEXT: s_lshl_b32 s89, s63, 24
-; SI-NEXT: s_lshl_b32 s90, s59, 24
-; SI-NEXT: s_lshl_b32 s91, s47, 24
-; SI-NEXT: s_lshl_b32 s92, s42, 24
-; SI-NEXT: s_lshl_b32 s93, s14, 24
-; SI-NEXT: s_lshl_b32 s94, s10, 24
-; SI-NEXT: s_lshl_b32 s95, s6, 24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v7, s4, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_or_b32_e32 v11, v7, v6
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v63
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v7, v18, v7
-; SI-NEXT: v_mov_b32_e32 v6, v13
-; SI-NEXT: v_or_b32_e32 v13, v5, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v62
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v60
-; SI-NEXT: v_or_b32_e32 v5, v5, v19
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v14, v7
-; SI-NEXT: v_mov_b32_e32 v60, v14
-; SI-NEXT: v_or_b32_e32 v14, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v57
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v0, v2, v0
+; SI-NEXT: v_and_b32_e32 v27, 0xff, v27
+; SI-NEXT: v_and_b32_e32 v22, 0xff, v22
+; SI-NEXT: v_or_b32_e32 v23, v26, v23
+; SI-NEXT: v_or_b32_e32 v26, v1, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v55
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v61
+; SI-NEXT: v_or_b32_e32 v27, v27, v58
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
+; SI-NEXT: v_or_b32_e32 v22, v30, v22
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v62, v1
+; SI-NEXT: v_or_b32_e32 v22, v27, v22
+; SI-NEXT: v_or_b32_e32 v27, v0, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v59
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v29, v1
+; SI-NEXT: v_or_b32_e32 v28, v0, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
+; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s5, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s77, s19, 24
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s5, s77, s5
; SI-NEXT: s_or_b32 s4, s4, s5
@@ -54208,6 +54273,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s5, s5, s77
; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s77, s78, s77
; SI-NEXT: s_or_b32 s5, s5, s77
@@ -54216,38 +54282,48 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s26, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
+; SI-NEXT: s_lshl_b32 s79, s27, 24
; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_or_b32 s78, s79, s78
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s28, 0xff
; SI-NEXT: s_lshl_b32 s79, s29, 8
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_mov_b32_e32 v51, v33
+; SI-NEXT: v_or_b32_e32 v13, v13, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s76, 0xff
+; SI-NEXT: s_and_b32 s79, s75, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s78, s78, 0xffff
; SI-NEXT: s_or_b32 s79, s88, s79
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s74, 0xff
-; SI-NEXT: s_lshl_b32 s88, s73, 8
+; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_lshl_b32 s88, s72, 8
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s72, 0xff
+; SI-NEXT: s_and_b32 s88, s63, 0xff
; SI-NEXT: s_lshl_b32 s88, s88, 16
+; SI-NEXT: s_lshl_b32 s89, s62, 24
; SI-NEXT: s_and_b32 s79, s79, 0xffff
; SI-NEXT: s_or_b32 s88, s89, s88
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s62, 0xff
-; SI-NEXT: s_lshl_b32 s89, s61, 8
+; SI-NEXT: s_and_b32 s88, s61, 0xff
+; SI-NEXT: s_lshl_b32 s89, s60, 8
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s60, 0xff
+; SI-NEXT: s_and_b32 s89, s59, 0xff
; SI-NEXT: s_lshl_b32 s89, s89, 16
+; SI-NEXT: s_lshl_b32 s90, s58, 24
; SI-NEXT: s_and_b32 s88, s88, 0xffff
; SI-NEXT: s_or_b32 s89, s90, s89
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s58, 0xff
-; SI-NEXT: s_lshl_b32 s90, s57, 8
+; SI-NEXT: s_and_b32 s89, s57, 0xff
+; SI-NEXT: s_lshl_b32 s90, s56, 8
; SI-NEXT: s_or_b32 s89, s89, s90
-; SI-NEXT: s_and_b32 s90, s56, 0xff
+; SI-NEXT: s_and_b32 s90, s47, 0xff
; SI-NEXT: s_lshl_b32 s90, s90, 16
+; SI-NEXT: s_lshl_b32 s91, s46, 24
; SI-NEXT: s_and_b32 s89, s89, 0xffff
; SI-NEXT: s_or_b32 s90, s91, s90
; SI-NEXT: s_or_b32 s89, s89, s90
@@ -54256,6 +54332,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s90, s90, s91
; SI-NEXT: s_and_b32 s91, s43, 0xff
; SI-NEXT: s_lshl_b32 s91, s91, 16
+; SI-NEXT: s_lshl_b32 s92, s42, 24
; SI-NEXT: s_and_b32 s90, s90, 0xffff
; SI-NEXT: s_or_b32 s91, s92, s91
; SI-NEXT: s_or_b32 s90, s90, s91
@@ -54264,6 +54341,8 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s91, s91, s92
; SI-NEXT: s_and_b32 s92, s15, 0xff
; SI-NEXT: s_lshl_b32 s92, s92, 16
+; SI-NEXT: s_lshl_b32 s93, s14, 24
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
; SI-NEXT: s_and_b32 s91, s91, 0xffff
; SI-NEXT: s_or_b32 s92, s93, s92
; SI-NEXT: s_or_b32 s91, s91, s92
@@ -54272,180 +54351,178 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s11, 0xff
; SI-NEXT: s_lshl_b32 s93, s93, 16
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v51, v7
-; SI-NEXT: v_or_b32_e32 v15, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v32
+; SI-NEXT: s_lshl_b32 s94, s10, 24
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: s_and_b32 s92, s92, 0xffff
; SI-NEXT: s_or_b32 s93, s94, s93
+; SI-NEXT: v_or_b32_e32 v8, v8, v24
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s9, 0xff
; SI-NEXT: s_lshl_b32 s94, s8, 8
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
+; SI-NEXT: v_or_b32_e32 v5, v9, v5
; SI-NEXT: s_or_b32 s93, s93, s94
; SI-NEXT: s_and_b32 s94, s7, 0xff
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v21, 0xff, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xff, v25
+; SI-NEXT: v_or_b32_e32 v24, v8, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v6
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_mov_b32_e32 v50, v33
+; SI-NEXT: v_or_b32_e32 v14, v33, v14
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v13, v13, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v47
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: s_lshl_b32 s94, s94, 16
+; SI-NEXT: s_lshl_b32 s95, s6, 24
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_or_b32_e32 v25, v25, v39
+; SI-NEXT: v_or_b32_e32 v5, v5, v10
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: s_and_b32 s93, s93, 0xffff
; SI-NEXT: s_or_b32 s94, s95, s94
-; SI-NEXT: s_or_b32 s93, s93, s94
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v45
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v48, v7
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_or_b32_e32 v16, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v43
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v52
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v44, v7
-; SI-NEXT: v_or_b32_e32 v17, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v41
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v21, v63, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v25
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v18, v5, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v5, s88
-; SI-NEXT: v_mov_b32_e32 v7, s90
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v19, v3, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v3, v7, v3
+; SI-NEXT: s_or_b32 s93, s93, s94
+; SI-NEXT: v_or_b32_e32 v23, v25, v23
+; SI-NEXT: v_or_b32_e32 v25, v5, v3
+; SI-NEXT: v_mov_b32_e32 v2, s77
; SI-NEXT: v_mov_b32_e32 v3, s78
; SI-NEXT: v_mov_b32_e32 v4, s79
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v20, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v54
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v5, s88
+; SI-NEXT: v_mov_b32_e32 v6, s89
+; SI-NEXT: v_mov_b32_e32 v7, s90
+; SI-NEXT: v_mov_b32_e32 v8, s91
+; SI-NEXT: v_mov_b32_e32 v9, s92
+; SI-NEXT: v_mov_b32_e32 v10, s93
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v21, v1
-; SI-NEXT: v_or_b32_e32 v21, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v50
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v49
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v22, v1
-; SI-NEXT: v_or_b32_e32 v22, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v39
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v23
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v23, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v27
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v48, v33
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v46
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
+; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_or_b32_e32 v33, v42, v33
+; SI-NEXT: v_mov_b32_e32 v46, v44
+; SI-NEXT: v_mov_b32_e32 v44, v42
+; SI-NEXT: v_mov_b32_e32 v42, v38
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_mov_b32_e32 v38, v40
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v40
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v33, v33, v45
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v24
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v55, v1
-; SI-NEXT: v_or_b32_e32 v24, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v59, v1
-; SI-NEXT: v_or_b32_e32 v25, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v29
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v26
-; SI-NEXT: v_or_b32_e32 v0, v0, v31
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v56, v1
-; SI-NEXT: v_or_b32_e32 v26, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v30
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_and_b32_e32 v1, 0xff, v35
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v27, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v61
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v28, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v6
-; SI-NEXT: v_mov_b32_e32 v6, s89
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_or_b32_e32 v1, v43, v1
; SI-NEXT: v_or_b32_e32 v29, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v10
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v0, v0, v46
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v32
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v34
+; SI-NEXT: v_or_b32_e32 v0, v0, v37
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v36, v1
; SI-NEXT: v_or_b32_e32 v30, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v8
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v40
-; SI-NEXT: v_or_b32_e32 v0, v0, v34
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v42
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v52
+; SI-NEXT: v_or_b32_e32 v0, v0, v46
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v37, v1
+; SI-NEXT: v_or_b32_e32 v1, v54, v1
; SI-NEXT: v_or_b32_e32 v31, v0, v1
; SI-NEXT: v_mov_b32_e32 v0, s4
; SI-NEXT: v_mov_b32_e32 v1, s5
-; SI-NEXT: v_mov_b32_e32 v2, s77
-; SI-NEXT: v_mov_b32_e32 v8, s91
-; SI-NEXT: v_mov_b32_e32 v9, s92
-; SI-NEXT: v_mov_b32_e32 v10, s93
; SI-NEXT: s_mov_b64 s[4:5], 0
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v15, v15, v40
+; SI-NEXT: v_or_b32_e32 v15, v33, v15
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v16, v40
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v33
+; SI-NEXT: v_or_b32_e32 v33, v33, v41
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v17, v17, v40
+; SI-NEXT: v_or_b32_e32 v17, v33, v17
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v18
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v18, v18, v40
+; SI-NEXT: v_or_b32_e32 v18, v33, v18
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v19, v19, v40
+; SI-NEXT: v_or_b32_e32 v19, v33, v19
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v20, v20, v40
+; SI-NEXT: v_or_b32_e32 v20, v33, v20
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: v_or_b32_e32 v21, v33, v21
; SI-NEXT: s_branch .LBB39_3
; SI-NEXT: .LBB39_2:
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
-; SI-NEXT: v_mov_b32_e32 v60, v14
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15_vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23_vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31
; SI-NEXT: .LBB39_3: ; %Flow
@@ -54454,9 +54531,9 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB39_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
-; SI-NEXT: s_add_i32 s46, s46, 3
-; SI-NEXT: s_and_b32 s4, s46, 0xff
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; SI-NEXT: s_add_i32 s76, s76, 3
+; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
@@ -54467,14 +54544,14 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s76, s76, 3
-; SI-NEXT: s_add_i32 s74, s74, 3
-; SI-NEXT: s_add_i32 s72, s72, 3
-; SI-NEXT: s_add_i32 s62, s62, 3
-; SI-NEXT: s_add_i32 s60, s60, 3
-; SI-NEXT: s_add_i32 s58, s58, 3
-; SI-NEXT: s_add_i32 s56, s56, 3
-; SI-NEXT: s_and_b32 s22, s56, 0xff
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s63, s63, 3
+; SI-NEXT: s_add_i32 s61, s61, 3
+; SI-NEXT: s_add_i32 s59, s59, 3
+; SI-NEXT: s_add_i32 s57, s57, 3
+; SI-NEXT: s_add_i32 s47, s47, 3
+; SI-NEXT: s_and_b32 s22, s47, 0xff
; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_add_i32 s45, s45, 3
; SI-NEXT: s_add_i32 s43, s43, 3
@@ -54512,22 +54589,22 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
; SI-NEXT: v_mov_b32_e32 v9, s10
; SI-NEXT: v_mov_b32_e32 v10, s6
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, s4, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -54559,38 +54636,38 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_and_b32 s17, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s29, 8
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s19, s76, 0xff
+; SI-NEXT: s_and_b32 s19, s75, 0xff
; SI-NEXT: s_addk_i32 s17, 0x300
-; SI-NEXT: s_lshl_b32 s18, s75, 24
+; SI-NEXT: s_lshl_b32 s18, s74, 24
; SI-NEXT: s_lshl_b32 s19, s19, 16
; SI-NEXT: s_and_b32 s17, s17, 0xffff
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s18, s74, 0xff
-; SI-NEXT: s_lshl_b32 s19, s73, 8
+; SI-NEXT: s_and_b32 s18, s73, 0xff
+; SI-NEXT: s_lshl_b32 s19, s72, 8
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_and_b32 s20, s63, 0xff
; SI-NEXT: s_addk_i32 s18, 0x300
-; SI-NEXT: s_lshl_b32 s19, s63, 24
+; SI-NEXT: s_lshl_b32 s19, s62, 24
; SI-NEXT: s_lshl_b32 s20, s20, 16
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s19, s62, 0xff
-; SI-NEXT: s_lshl_b32 s20, s61, 8
+; SI-NEXT: s_and_b32 s19, s61, 0xff
+; SI-NEXT: s_lshl_b32 s20, s60, 8
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s21, s60, 0xff
+; SI-NEXT: s_and_b32 s21, s59, 0xff
; SI-NEXT: s_addk_i32 s19, 0x300
-; SI-NEXT: s_lshl_b32 s20, s59, 24
+; SI-NEXT: s_lshl_b32 s20, s58, 24
; SI-NEXT: s_lshl_b32 s21, s21, 16
; SI-NEXT: s_and_b32 s19, s19, 0xffff
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s20, s58, 0xff
-; SI-NEXT: s_lshl_b32 s21, s57, 8
+; SI-NEXT: s_and_b32 s20, s57, 0xff
+; SI-NEXT: s_lshl_b32 s21, s56, 8
; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_addk_i32 s20, 0x300
-; SI-NEXT: s_lshl_b32 s21, s47, 24
+; SI-NEXT: s_lshl_b32 s21, s46, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_or_b32 s20, s21, s20
@@ -54610,7 +54687,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_or_b32_e32 v2, v2, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x300, v2
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: s_or_b32 s22, s23, s22
@@ -54638,333 +54715,330 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: v_or_b32_e32 v0, v3, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; SI-NEXT: v_mov_b32_e32 v3, s17
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v38, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v51, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v63, v0
+; SI-NEXT: v_or_b32_e32 v0, v50, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v62, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v48, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v60, v0
+; SI-NEXT: v_or_b32_e32 v0, v44, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
+; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v38
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v16, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v17, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v18, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v20, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v21, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v22, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v24, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v25, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v26, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v27, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v28, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v29, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -54974,21 +55048,21 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v30, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -55013,7 +55087,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0)
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
;
; VI-LABEL: bitcast_v128i8_to_v32f32_scalar:
@@ -55038,8 +55112,9 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; VI-NEXT: v_readfirstlane_b32 s73, v3
+; VI-NEXT: v_readfirstlane_b32 s74, v2
; VI-NEXT: v_readfirstlane_b32 s47, v13
+; VI-NEXT: v_readfirstlane_b32 s72, v4
; VI-NEXT: v_readfirstlane_b32 s61, v7
; VI-NEXT: v_readfirstlane_b32 s62, v6
; VI-NEXT: v_readfirstlane_b32 s63, v5
@@ -55065,101 +55140,100 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
-; VI-NEXT: v_readfirstlane_b32 s72, v4
-; VI-NEXT: v_readfirstlane_b32 s74, v2
+; VI-NEXT: v_readfirstlane_b32 s73, v3
; VI-NEXT: v_readfirstlane_b32 s75, v1
-; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
+; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:316
+; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
+; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:308
; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:296
; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; VI-NEXT: s_waitcnt vmcnt(13)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: s_waitcnt vmcnt(10)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:320
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:268
+; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:264
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:248
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
-; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:220
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; VI-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; VI-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; VI-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; VI-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:180
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:148
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:136
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -55180,186 +55254,192 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:48
; VI-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; VI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; VI-NEXT: s_cbranch_scc0 .LBB39_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: v_mov_b32_e32 v20, 0xc0c0004
-; VI-NEXT: v_mov_b32_e32 v4, s23
-; VI-NEXT: v_mov_b32_e32 v2, s19
-; VI-NEXT: v_mov_b32_e32 v3, s21
-; VI-NEXT: v_perm_b32 v4, s22, v4, v20
+; VI-NEXT: v_mov_b32_e32 v40, 0xc0c0004
+; VI-NEXT: v_mov_b32_e32 v3, s19
; VI-NEXT: v_mov_b32_e32 v13, s17
-; VI-NEXT: v_perm_b32 v2, s18, v2, v20
-; VI-NEXT: v_perm_b32 v3, s20, v3, v20
-; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_perm_b32 v13, s16, v13, v20
+; VI-NEXT: v_perm_b32 v3, s18, v3, v40
+; VI-NEXT: v_mov_b32_e32 v2, s23
+; VI-NEXT: v_perm_b32 v13, s16, v13, v40
+; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; VI-NEXT: v_mov_b32_e32 v42, s21
+; VI-NEXT: v_perm_b32 v2, s22, v2, v40
+; VI-NEXT: v_perm_b32 v42, s20, v42, v40
; VI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; VI-NEXT: v_mov_b32_e32 v19, v1
-; VI-NEXT: v_or_b32_e32 v1, v3, v4
+; VI-NEXT: v_mov_b32_e32 v20, v0
+; VI-NEXT: v_or_b32_e32 v0, v13, v3
; VI-NEXT: v_mov_b32_e32 v3, s27
-; VI-NEXT: v_mov_b32_e32 v22, v0
-; VI-NEXT: v_or_b32_e32 v0, v13, v2
+; VI-NEXT: v_mov_b32_e32 v19, v1
+; VI-NEXT: v_or_b32_e32 v1, v42, v2
; VI-NEXT: v_mov_b32_e32 v2, s25
-; VI-NEXT: v_perm_b32 v3, s26, v3, v20
-; VI-NEXT: v_perm_b32 v2, s24, v2, v20
+; VI-NEXT: v_perm_b32 v3, s26, v3, v40
+; VI-NEXT: v_mov_b32_e32 v59, v4
+; VI-NEXT: v_perm_b32 v2, s24, v2, v40
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; VI-NEXT: v_mov_b32_e32 v4, s75
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s76, v4, v20
-; VI-NEXT: v_mov_b32_e32 v44, v5
-; VI-NEXT: v_perm_b32 v3, s28, v3, v20
+; VI-NEXT: v_perm_b32 v4, s76, v4, v40
+; VI-NEXT: v_mov_b32_e32 v47, v5
+; VI-NEXT: v_perm_b32 v3, s28, v3, v40
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; VI-NEXT: v_mov_b32_e32 v5, s63
; VI-NEXT: v_or_b32_e32 v3, v3, v4
; VI-NEXT: v_mov_b32_e32 v4, s73
-; VI-NEXT: v_perm_b32 v5, s72, v5, v20
-; VI-NEXT: v_mov_b32_e32 v46, v6
-; VI-NEXT: v_perm_b32 v4, s74, v4, v20
+; VI-NEXT: v_perm_b32 v5, s72, v5, v40
+; VI-NEXT: v_mov_b32_e32 v57, v6
+; VI-NEXT: v_perm_b32 v4, s74, v4, v40
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; VI-NEXT: v_mov_b32_e32 v6, s59
; VI-NEXT: v_or_b32_e32 v4, v4, v5
; VI-NEXT: v_mov_b32_e32 v5, s61
-; VI-NEXT: v_perm_b32 v6, s60, v6, v20
-; VI-NEXT: v_mov_b32_e32 v45, v7
-; VI-NEXT: v_perm_b32 v5, s62, v5, v20
+; VI-NEXT: v_perm_b32 v6, s60, v6, v40
+; VI-NEXT: v_mov_b32_e32 v56, v7
+; VI-NEXT: v_perm_b32 v5, s62, v5, v40
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; VI-NEXT: v_mov_b32_e32 v7, s47
; VI-NEXT: v_or_b32_e32 v5, v5, v6
; VI-NEXT: v_mov_b32_e32 v6, s57
-; VI-NEXT: v_perm_b32 v7, s56, v7, v20
-; VI-NEXT: v_mov_b32_e32 v47, v40
-; VI-NEXT: v_mov_b32_e32 v40, v8
-; VI-NEXT: v_perm_b32 v6, s58, v6, v20
+; VI-NEXT: v_perm_b32 v7, s56, v7, v40
+; VI-NEXT: v_mov_b32_e32 v43, v8
+; VI-NEXT: v_perm_b32 v6, s58, v6, v40
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_mov_b32_e32 v7, s45
-; VI-NEXT: v_perm_b32 v8, s44, v8, v20
-; VI-NEXT: v_mov_b32_e32 v43, v9
-; VI-NEXT: v_perm_b32 v7, s46, v7, v20
+; VI-NEXT: v_perm_b32 v8, s44, v8, v40
+; VI-NEXT: v_mov_b32_e32 v46, v9
+; VI-NEXT: v_perm_b32 v7, s46, v7, v40
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s15
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s41
-; VI-NEXT: v_perm_b32 v9, s40, v9, v20
-; VI-NEXT: v_mov_b32_e32 v56, v41
-; VI-NEXT: v_mov_b32_e32 v41, v10
-; VI-NEXT: v_perm_b32 v8, s42, v8, v20
+; VI-NEXT: v_perm_b32 v9, s40, v9, v40
+; VI-NEXT: v_mov_b32_e32 v44, v10
+; VI-NEXT: v_perm_b32 v8, s42, v8, v40
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_or_b32_e32 v8, v8, v9
; VI-NEXT: v_mov_b32_e32 v9, s13
-; VI-NEXT: v_perm_b32 v10, s12, v10, v20
-; VI-NEXT: v_mov_b32_e32 v42, v11
-; VI-NEXT: v_perm_b32 v9, s14, v9, v20
+; VI-NEXT: v_perm_b32 v10, s12, v10, v40
+; VI-NEXT: v_mov_b32_e32 v58, v41
+; VI-NEXT: v_mov_b32_e32 v41, v11
+; VI-NEXT: v_perm_b32 v9, s14, v9, v40
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_or_b32_e32 v9, v9, v10
; VI-NEXT: v_mov_b32_e32 v10, s9
-; VI-NEXT: v_perm_b32 v11, s8, v11, v20
-; VI-NEXT: v_perm_b32 v10, s10, v10, v20
+; VI-NEXT: v_perm_b32 v11, s8, v11, v40
+; VI-NEXT: v_perm_b32 v10, s10, v10, v40
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
+; VI-NEXT: v_perm_b32 v18, v41, v18, s4
+; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; VI-NEXT: v_perm_b32 v19, v63, v19, s4
+; VI-NEXT: v_or_b32_e32 v18, v19, v18
+; VI-NEXT: v_perm_b32 v19, v22, v45, s4
+; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; VI-NEXT: v_perm_b32 v20, v62, v20, s4
+; VI-NEXT: v_or_b32_e32 v19, v20, v19
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: v_perm_b32 v12, v12, v61, s4
; VI-NEXT: v_perm_b32 v14, v52, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_perm_b32 v15, v48, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_perm_b32 v16, v50, v16, s4
-; VI-NEXT: v_perm_b32 v18, v42, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_perm_b32 v17, v35, v17, s4
-; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
-; VI-NEXT: v_perm_b32 v19, v63, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
-; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_perm_b32 v19, v23, v24, s4
-; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; VI-NEXT: v_perm_b32 v23, v34, v33, s4
-; VI-NEXT: v_perm_b32 v24, v41, v43, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v11, s6, v11, v20
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v22, v30, v31, s4
+; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: v_perm_b32 v11, s6, v11, v40
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: s_waitcnt vmcnt(2)
+; VI-NEXT: v_perm_b32 v20, v25, v20, s4
+; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v25, v56, v57, s4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v20, v13, s4
+; VI-NEXT: v_perm_b32 v13, v40, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_e32 v11, v11, v13
; VI-NEXT: v_perm_b32 v13, v21, v60, s4
@@ -55367,88 +55447,84 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v27, v29, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; VI-NEXT: v_perm_b32 v20, v57, v28, s4
-; VI-NEXT: v_or_b32_e32 v13, v20, v13
-; VI-NEXT: v_perm_b32 v20, v55, v51, s4
-; VI-NEXT: v_or_b32_e32 v14, v20, v14
-; VI-NEXT: v_perm_b32 v20, v54, v53, s4
-; VI-NEXT: v_or_b32_e32 v15, v20, v15
-; VI-NEXT: v_perm_b32 v20, v39, v36, s4
-; VI-NEXT: v_or_b32_e32 v16, v20, v16
-; VI-NEXT: v_perm_b32 v20, v49, v38, s4
-; VI-NEXT: v_or_b32_e32 v17, v20, v17
-; VI-NEXT: v_perm_b32 v20, v62, v22, s4
-; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
-; VI-NEXT: v_perm_b32 v21, v25, v58, s4
-; VI-NEXT: v_perm_b32 v22, v30, v31, s4
-; VI-NEXT: v_perm_b32 v25, v45, v46, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v26, v20, s4
-; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v21, v26, v28, s4
+; VI-NEXT: v_or_b32_e32 v13, v21, v13
+; VI-NEXT: v_perm_b32 v21, v55, v32, s4
+; VI-NEXT: v_or_b32_e32 v14, v21, v14
+; VI-NEXT: v_perm_b32 v21, v54, v53, s4
+; VI-NEXT: v_or_b32_e32 v15, v21, v15
+; VI-NEXT: v_perm_b32 v21, v39, v36, s4
+; VI-NEXT: v_or_b32_e32 v16, v21, v16
+; VI-NEXT: v_perm_b32 v21, v49, v38, s4
+; VI-NEXT: v_or_b32_e32 v17, v21, v17
+; VI-NEXT: v_perm_b32 v21, v23, v24, s4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v23, v34, v33, s4
+; VI-NEXT: v_perm_b32 v24, v44, v46, s4
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v37, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v32, v22, s4
+; VI-NEXT: v_perm_b32 v22, v51, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v40, v23, s4
+; VI-NEXT: v_perm_b32 v23, v43, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v44, v24, s4
+; VI-NEXT: v_perm_b32 v24, v47, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v25, v26, s4
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v25
+; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v25
; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v26, v25, s4
-; VI-NEXT: v_or_b32_e32 v25, v25, v27
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v25, v59, v25, s4
+; VI-NEXT: v_or_b32_e32 v25, v25, v26
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v26, v26, v27, s4
+; VI-NEXT: v_perm_b32 v26, v26, v58, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_or_b32_e32 v26, v26, v28
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v27, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; VI-NEXT: v_or_b32_e32 v27, v27, v29
-; VI-NEXT: v_perm_b32 v29, v47, v56, s4
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v28, v59, v28, s4
+; VI-NEXT: v_perm_b32 v28, v29, v28, s4
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v28, v29, v28
; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
-; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v30, v31, v30, s4
@@ -55463,16 +55539,16 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v31, v32, v31, s4
; VI-NEXT: v_or_b32_e32 v30, v31, v30
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v31, v32, v31, s4
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v31, v31, v32, s4
+; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v31
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v32, v33, v32, s4
-; VI-NEXT: v_or_b32_e32 v31, v32, v31
+; VI-NEXT: v_perm_b32 v31, v33, v31, s4
+; VI-NEXT: v_or_b32_e32 v31, v31, v32
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB39_3
; VI-NEXT: .LBB39_2:
@@ -55584,7 +55660,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x300, v10
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_add_u32_e32 v0, vcc, 0x3000000, v0
@@ -55599,12 +55675,12 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_add_u32_e32 v9, vcc, 0x3000000, v9
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x3000000, v10
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v12, s6, v11, v12
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v12
+; VI-NEXT: v_perm_b32 v11, s6, v11, v12
+; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v11
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v11, v13, v11, s4
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
@@ -55615,57 +55691,57 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v12
; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v13, v12, s4
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
+; VI-NEXT: v_perm_b32 v12, v13, v12, s4
+; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v12
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v12, v14, v12, s4
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v12, v12, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v12, vcc, 0x3000000, v12
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v14, v14, v13, s4
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
+; VI-NEXT: v_perm_b32 v13, v14, v13, s4
+; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v13
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v15, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x3000000, v13
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v15, v15, v14, s4
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v15
+; VI-NEXT: v_perm_b32 v14, v15, v14, s4
+; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v14
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v14, v16, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v14, v14, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x3000000, v14
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v15
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v16, v16, v15, s4
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v16
+; VI-NEXT: v_perm_b32 v15, v16, v15, s4
+; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v15
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v15
; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
@@ -55673,151 +55749,151 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_perm_b32 v15, v17, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_or_b32_sdwa v15, v15, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v15, vcc, 0x3000000, v15
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v17, v17, v16, s4
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v17
+; VI-NEXT: v_perm_b32 v16, v17, v16, s4
+; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v16
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v16, v18, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_or_b32_sdwa v16, v16, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v16, vcc, 0x3000000, v16
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v18, v18, v17, s4
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v18
+; VI-NEXT: v_perm_b32 v17, v18, v17, s4
+; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v17
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v17, v19, v17, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; VI-NEXT: v_or_b32_sdwa v17, v17, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v17, vcc, 0x3000000, v17
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v19, v19, v18, s4
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v19
+; VI-NEXT: v_perm_b32 v18, v19, v18, s4
+; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v18
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v18, v20, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; VI-NEXT: v_or_b32_sdwa v18, v18, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v18, vcc, 0x3000000, v18
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v20, v19, s4
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v20
+; VI-NEXT: v_perm_b32 v19, v20, v19, s4
+; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v19
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v19, v21, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; VI-NEXT: v_or_b32_sdwa v19, v19, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v19, vcc, 0x3000000, v19
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v21, v21, v20, s4
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v21
+; VI-NEXT: v_perm_b32 v20, v21, v20, s4
+; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v20
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v20, v22, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; VI-NEXT: v_or_b32_sdwa v20, v20, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v20, vcc, 0x3000000, v20
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v22, v21, s4
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v22
+; VI-NEXT: v_perm_b32 v21, v22, v21, s4
+; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v21
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v23, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_sdwa v21, v21, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v21, vcc, 0x3000000, v21
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v23, v22, s4
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v23
+; VI-NEXT: v_perm_b32 v22, v23, v22, s4
+; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v22
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v22, v24, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_sdwa v22, v22, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v22, vcc, 0x3000000, v22
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v24, v23, s4
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v24
+; VI-NEXT: v_perm_b32 v23, v24, v23, s4
+; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v23
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v23, v25, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_sdwa v23, v23, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v23, vcc, 0x3000000, v23
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v25, v24, s4
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v25
+; VI-NEXT: v_perm_b32 v24, v25, v24, s4
+; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v24
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v24, v26, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_sdwa v24, v24, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v24, vcc, 0x3000000, v24
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v25
@@ -55825,55 +55901,55 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v26, v25, s4
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x300, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v27, v25, s4
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_or_b32_sdwa v25, v25, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v25, vcc, 0x3000000, v25
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x300, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; VI-NEXT: v_or_b32_sdwa v26, v26, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x3000000, v26
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x300, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v29, v27, s4
; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; VI-NEXT: v_or_b32_sdwa v27, v27, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x3000000, v27
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v28, v29, v28, s4
; VI-NEXT: v_add_u32_e32 v29, vcc, 0x300, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v28
; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
@@ -55881,11 +55957,11 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_perm_b32 v28, v30, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; VI-NEXT: v_or_b32_sdwa v28, v28, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x3000000, v28
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v29
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_add_u32_e32 v30, vcc, 0x300, v29
@@ -55972,8 +56048,9 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; GFX9-NEXT: v_readfirstlane_b32 s73, v3
-; GFX9-NEXT: v_readfirstlane_b32 s47, v13
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s72, v4
+; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s61, v7
; GFX9-NEXT: v_readfirstlane_b32 s62, v6
; GFX9-NEXT: v_readfirstlane_b32 s63, v5
@@ -55990,31 +56067,28 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_readfirstlane_b32 s13, v23
; GFX9-NEXT: v_readfirstlane_b32 s14, v22
; GFX9-NEXT: v_readfirstlane_b32 s15, v21
-; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s41, v19
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
+; GFX9-NEXT: v_readfirstlane_b32 s47, v13
; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
-; GFX9-NEXT: v_readfirstlane_b32 s72, v4
-; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s73, v3
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
-; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
+; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:320
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:316
+; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:296
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; GFX9-NEXT: s_waitcnt vmcnt(13)
+; GFX9-NEXT: s_waitcnt vmcnt(11)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -56024,82 +56098,86 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; GFX9-NEXT: s_nop 0
+; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:264
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:248
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
+; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; GFX9-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; GFX9-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; GFX9-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
-; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
+; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:176
; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; GFX9-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:148
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:136
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -56107,108 +56185,109 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:108
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:104
-; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:100
; GFX9-NEXT: buffer_load_ushort v38, off, s[0:3], s32 offset:96
-; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v50, off, s[0:3], s32 offset:80
; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v15, off, s[0:3], s32 offset:72
-; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:68
-; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:64
-; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
-; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
+; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:68
+; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:64
+; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:60
+; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:56
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(55)
-; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(54)
-; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(49)
-; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(62)
; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
@@ -56228,7 +56307,8 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v19, v1
; GFX9-NEXT: v_or_b32_e32 v1, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v3, s27
-; GFX9-NEXT: v_mov_b32_e32 v22, v0
+; GFX9-NEXT: v_mov_b32_e32 v46, v42
+; GFX9-NEXT: v_mov_b32_e32 v42, v0
; GFX9-NEXT: v_or_b32_e32 v0, v13, v2
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v20
@@ -56238,28 +56318,28 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_e32 v2, v2, v3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v20
-; GFX9-NEXT: v_mov_b32_e32 v44, v5
+; GFX9-NEXT: v_mov_b32_e32 v45, v5
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s63
; GFX9-NEXT: v_or_b32_e32 v3, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v4, s73
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v20
-; GFX9-NEXT: v_mov_b32_e32 v46, v6
+; GFX9-NEXT: v_mov_b32_e32 v56, v6
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s59
; GFX9-NEXT: v_or_b32_e32 v4, v4, v5
; GFX9-NEXT: v_mov_b32_e32 v5, s61
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v20
-; GFX9-NEXT: v_mov_b32_e32 v45, v7
+; GFX9-NEXT: v_mov_b32_e32 v47, v7
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s47
; GFX9-NEXT: v_or_b32_e32 v5, v5, v6
; GFX9-NEXT: v_mov_b32_e32 v6, s57
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v20
-; GFX9-NEXT: v_mov_b32_e32 v47, v40
+; GFX9-NEXT: v_mov_b32_e32 v57, v40
; GFX9-NEXT: v_mov_b32_e32 v40, v8
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
@@ -56267,22 +56347,21 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_e32 v6, v6, v7
; GFX9-NEXT: v_mov_b32_e32 v7, s45
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v20
-; GFX9-NEXT: v_mov_b32_e32 v43, v9
+; GFX9-NEXT: v_mov_b32_e32 v44, v9
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v9, s15
; GFX9-NEXT: v_or_b32_e32 v7, v7, v8
; GFX9-NEXT: v_mov_b32_e32 v8, s41
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v20
-; GFX9-NEXT: v_mov_b32_e32 v56, v41
-; GFX9-NEXT: v_mov_b32_e32 v41, v10
+; GFX9-NEXT: v_mov_b32_e32 v43, v10
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; GFX9-NEXT: v_mov_b32_e32 v10, s11
; GFX9-NEXT: v_or_b32_e32 v8, v8, v9
; GFX9-NEXT: v_mov_b32_e32 v9, s13
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v20
-; GFX9-NEXT: v_mov_b32_e32 v42, v11
+; GFX9-NEXT: v_mov_b32_e32 v25, v11
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: v_mov_b32_e32 v11, s7
@@ -56295,22 +56374,21 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v12, v12, v61, s4
-; GFX9-NEXT: v_perm_b32 v14, v52, v14, s4
-; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v32, v15, s4
+; GFX9-NEXT: v_perm_b32 v14, v33, v14, s4
+; GFX9-NEXT: v_perm_b32 v15, v39, v15, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; GFX9-NEXT: v_perm_b32 v16, v53, v16, s4
-; GFX9-NEXT: v_perm_b32 v18, v42, v18, s4
+; GFX9-NEXT: v_perm_b32 v18, v25, v18, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; GFX9-NEXT: v_perm_b32 v17, v35, v17, s4
+; GFX9-NEXT: v_perm_b32 v17, v49, v17, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; GFX9-NEXT: v_perm_b32 v19, v63, v19, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; GFX9-NEXT: v_or_b32_e32 v18, v19, v18
-; GFX9-NEXT: v_perm_b32 v19, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v19, v22, v41, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; GFX9-NEXT: v_perm_b32 v23, v34, v33, s4
-; GFX9-NEXT: v_perm_b32 v24, v41, v43, s4
+; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
+; GFX9-NEXT: v_perm_b32 v25, v47, v56, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v11, s6, v11, v20
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
@@ -56324,81 +56402,83 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_e32 v12, v12, v13
; GFX9-NEXT: v_perm_b32 v13, v27, v29, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; GFX9-NEXT: v_perm_b32 v20, v57, v28, s4
+; GFX9-NEXT: v_perm_b32 v20, v26, v28, s4
; GFX9-NEXT: v_or_b32_e32 v13, v20, v13
-; GFX9-NEXT: v_perm_b32 v20, v55, v48, s4
-; GFX9-NEXT: v_or_b32_e32 v14, v20, v14
-; GFX9-NEXT: v_perm_b32 v20, v54, v39, s4
+; GFX9-NEXT: v_perm_b32 v20, v52, v48, s4
+; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; GFX9-NEXT: v_or_b32_e32 v14, v14, v20
+; GFX9-NEXT: v_perm_b32 v20, v32, v54, s4
; GFX9-NEXT: v_or_b32_e32 v15, v20, v15
; GFX9-NEXT: v_perm_b32 v20, v36, v50, s4
; GFX9-NEXT: v_or_b32_e32 v16, v20, v16
-; GFX9-NEXT: v_perm_b32 v20, v49, v38, s4
+; GFX9-NEXT: v_perm_b32 v20, v35, v38, s4
; GFX9-NEXT: v_or_b32_e32 v17, v20, v17
-; GFX9-NEXT: v_perm_b32 v20, v62, v22, s4
+; GFX9-NEXT: v_perm_b32 v20, v62, v42, s4
; GFX9-NEXT: v_or_b32_e32 v19, v20, v19
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v21, v25, v58, s4
-; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
-; GFX9-NEXT: v_perm_b32 v25, v45, v46, s4
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v21, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v23, v34, v55, s4
+; GFX9-NEXT: v_perm_b32 v24, v43, v44, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v26, v20, s4
+; GFX9-NEXT: v_perm_b32 v20, v58, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; GFX9-NEXT: v_or_b32_e32 v20, v21, v20
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v21, v37, v21, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; GFX9-NEXT: v_or_b32_e32 v21, v22, v21
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v22, v51, v22, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; GFX9-NEXT: v_or_b32_e32 v22, v23, v22
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v23, v40, v23, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; GFX9-NEXT: v_or_b32_e32 v23, v24, v23
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v44, v24, s4
+; GFX9-NEXT: v_perm_b32 v24, v45, v24, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; GFX9-NEXT: v_or_b32_e32 v24, v25, v24
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v26, s4
+; GFX9-NEXT: v_perm_b32 v25, v25, v46, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_or_b32_e32 v25, v25, v27
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v26, v27, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_or_b32_e32 v26, v26, v28
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v27, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: v_or_b32_e32 v27, v27, v29
-; GFX9-NEXT: v_perm_b32 v29, v47, v56, s4
-; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
; GFX9-NEXT: v_perm_b32 v28, v59, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_perm_b32 v29, v57, v29, s4
; GFX9-NEXT: v_or_b32_e32 v28, v29, v28
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
@@ -56420,10 +56500,10 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_or_b32_e32 v30, v31, v30
-; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
+; GFX9-NEXT: v_perm_b32 v31, v31, v32, s4
; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -56446,21 +56526,21 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v12, 0xc0c0004
; GFX9-NEXT: s_add_i32 s18, s18, 3
; GFX9-NEXT: v_mov_b32_e32 v1, s19
-; GFX9-NEXT: s_add_i32 s20, s20, 3
-; GFX9-NEXT: v_mov_b32_e32 v2, s21
-; GFX9-NEXT: s_add_i32 s22, s22, 3
-; GFX9-NEXT: v_mov_b32_e32 v3, s23
; GFX9-NEXT: v_perm_b32 v0, s16, v0, v12
-; GFX9-NEXT: s_movk_i32 s4, 0x300
+; GFX9-NEXT: s_movk_i32 s5, 0x300
; GFX9-NEXT: v_perm_b32 v1, s18, v1, v12
-; GFX9-NEXT: v_perm_b32 v2, s20, v2, v12
-; GFX9-NEXT: v_perm_b32 v3, s22, v3, v12
; GFX9-NEXT: v_add_u32_e32 v0, 0x300, v0
-; GFX9-NEXT: v_add_u32_sdwa v1, v1, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v1, v1, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_or_b32_sdwa v1, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX9-NEXT: s_add_i32 s20, s20, 3
+; GFX9-NEXT: v_mov_b32_e32 v1, s21
+; GFX9-NEXT: s_add_i32 s22, s22, 3
+; GFX9-NEXT: v_mov_b32_e32 v2, s23
+; GFX9-NEXT: v_perm_b32 v1, s20, v1, v12
+; GFX9-NEXT: v_perm_b32 v2, s22, v2, v12
+; GFX9-NEXT: v_add_u32_e32 v1, 0x300, v1
+; GFX9-NEXT: v_add_u32_sdwa v2, v2, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s24, s24, 3
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: s_add_i32 s26, s26, 3
@@ -56468,7 +56548,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v12
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v12
; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v3, v3, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v2, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s28, s28, 3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
@@ -56477,7 +56557,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v12
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v12
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
-; GFX9-NEXT: v_add_u32_sdwa v4, v4, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v3, v3, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_mov_b32_e32 v4, s73
@@ -56486,7 +56566,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v12
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v12
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
-; GFX9-NEXT: v_add_u32_sdwa v5, v5, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v4, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s62, s62, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s61
@@ -56495,7 +56575,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v12
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v12
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u32_sdwa v6, v6, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v5, v5, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s58, s58, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s57
@@ -56504,7 +56584,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v12
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v12
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
-; GFX9-NEXT: v_add_u32_sdwa v7, v7, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v6, v6, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s46, s46, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s45
@@ -56513,7 +56593,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v12
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v12
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
-; GFX9-NEXT: v_add_u32_sdwa v8, v8, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v8, v8, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v7, v7, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v8, s41
@@ -56522,7 +56602,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v12
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v12
; GFX9-NEXT: v_add_u32_e32 v8, 0x300, v8
-; GFX9-NEXT: v_add_u32_sdwa v9, v9, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v9, v9, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v8, v8, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s14, s14, 3
; GFX9-NEXT: v_mov_b32_e32 v9, s13
@@ -56531,7 +56611,7 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v12
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v12
; GFX9-NEXT: v_add_u32_e32 v9, 0x300, v9
-; GFX9-NEXT: v_add_u32_sdwa v10, v10, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v10, v10, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v9, v9, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s10, s10, 3
; GFX9-NEXT: v_mov_b32_e32 v10, s9
@@ -56540,321 +56620,321 @@ define inreg <32 x float> @bitcast_v128i8_to_v32f32_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v10, s10, v10, v12
; GFX9-NEXT: v_perm_b32 v11, s8, v11, v12
; GFX9-NEXT: v_add_u32_e32 v10, 0x300, v10
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_add_i32 s6, s6, 3
-; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
+; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, s6, v11, v12
+; GFX9-NEXT: v_perm_b32 v11, s6, v11, v12
+; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v12
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v11, v13, v11, s5
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v11, v13, v11, s4
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v12
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v13, v12, s5
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
+; GFX9-NEXT: v_perm_b32 v12, v13, v12, s4
+; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v12
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, v14, v12, s5
-; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v12, v14, v12, s4
+; GFX9-NEXT: v_add_u32_sdwa v12, v12, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v12, v13, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v14, v13, s5
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
+; GFX9-NEXT: v_perm_b32 v13, v14, v13, s4
+; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v13
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v15, v13, s5
-; GFX9-NEXT: v_add_u32_sdwa v13, v13, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v13, v15, v13, s4
+; GFX9-NEXT: v_add_u32_sdwa v13, v13, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v15, v14, s5
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v15
+; GFX9-NEXT: v_perm_b32 v14, v15, v14, s4
+; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v14
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v16, v14, s5
-; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v16, v14, s4
+; GFX9-NEXT: v_add_u32_sdwa v14, v14, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v16, v15, s5
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v16
+; GFX9-NEXT: v_perm_b32 v15, v16, v15, s4
+; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v15
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v17, v15, s5
-; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v15, v17, v15, s4
+; GFX9-NEXT: v_add_u32_sdwa v15, v15, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v15, v16, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v17, v16, s5
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v17
+; GFX9-NEXT: v_perm_b32 v16, v17, v16, s4
+; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v16
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v18, v16, s5
-; GFX9-NEXT: v_add_u32_sdwa v16, v16, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v16, v18, v16, s4
+; GFX9-NEXT: v_add_u32_sdwa v16, v16, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v16, v17, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v18, v17, s5
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v18
+; GFX9-NEXT: v_perm_b32 v17, v18, v17, s4
+; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v17
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v19, v17, s5
-; GFX9-NEXT: v_add_u32_sdwa v17, v17, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v17, v19, v17, s4
+; GFX9-NEXT: v_add_u32_sdwa v17, v17, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v17, v18, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v19, v18, s5
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v19
+; GFX9-NEXT: v_perm_b32 v18, v19, v18, s4
+; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v18
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v20, v18, s5
-; GFX9-NEXT: v_add_u32_sdwa v18, v18, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v18, v20, v18, s4
+; GFX9-NEXT: v_add_u32_sdwa v18, v18, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v18, v19, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v20, v19, s5
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v20
+; GFX9-NEXT: v_perm_b32 v19, v20, v19, s4
+; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v19
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v21, v19, s5
-; GFX9-NEXT: v_add_u32_sdwa v19, v19, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v19, v21, v19, s4
+; GFX9-NEXT: v_add_u32_sdwa v19, v19, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v19, v20, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v21, v20, s5
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v21
+; GFX9-NEXT: v_perm_b32 v20, v21, v20, s4
+; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v20
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v22, v20, s5
-; GFX9-NEXT: v_add_u32_sdwa v20, v20, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v20, v22, v20, s4
+; GFX9-NEXT: v_add_u32_sdwa v20, v20, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v20, v21, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v22, v21, s5
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v22
+; GFX9-NEXT: v_perm_b32 v21, v22, v21, s4
+; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v21
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v23, v21, s5
-; GFX9-NEXT: v_add_u32_sdwa v21, v21, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v21, v23, v21, s4
+; GFX9-NEXT: v_add_u32_sdwa v21, v21, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v21, v22, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v23, v22, s5
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v23
+; GFX9-NEXT: v_perm_b32 v22, v23, v22, s4
+; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v22
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v24, v22, s5
-; GFX9-NEXT: v_add_u32_sdwa v22, v22, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v22, v24, v22, s4
+; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v22, v23, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v24, v23, s5
+; GFX9-NEXT: v_perm_b32 v23, v24, v23, s4
+; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v23
; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v24
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v25, v23, s5
-; GFX9-NEXT: v_add_u32_sdwa v23, v23, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v23, v25, v23, s4
+; GFX9-NEXT: v_add_u32_sdwa v23, v23, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v23, v24, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v24, s5
+; GFX9-NEXT: v_perm_b32 v24, v25, v24, s4
+; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v24
; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v25
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v26, v24, s5
-; GFX9-NEXT: v_add_u32_sdwa v24, v24, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v24, v26, v24, s4
+; GFX9-NEXT: v_add_u32_sdwa v24, v24, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v24, v25, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v26, v25, s5
+; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_add_u32_e32 v26, 0x300, v25
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v27, v25, s5
-; GFX9-NEXT: v_add_u32_sdwa v25, v25, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v25, v27, v25, s4
+; GFX9-NEXT: v_add_u32_sdwa v25, v25, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v25, v26, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v27, v26, s5
+; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_add_u32_e32 v27, 0x300, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v28, v26, s5
-; GFX9-NEXT: v_add_u32_sdwa v26, v26, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v26, v28, v26, s4
+; GFX9-NEXT: v_add_u32_sdwa v26, v26, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v26, v27, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v28, v27, s5
+; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: v_add_u32_e32 v28, 0x300, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v29, v27, s5
-; GFX9-NEXT: v_add_u32_sdwa v27, v27, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v27, v29, v27, s4
+; GFX9-NEXT: v_add_u32_sdwa v27, v27, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v27, v28, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v28
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v29, v28, s5
+; GFX9-NEXT: v_perm_b32 v28, v29, v28, s4
; GFX9-NEXT: v_add_u32_e32 v29, 0x300, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v30, v28, s5
-; GFX9-NEXT: v_add_u32_sdwa v28, v28, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v28, v30, v28, s4
+; GFX9-NEXT: v_add_u32_sdwa v28, v28, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v28, v29, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v30, v29, s5
+; GFX9-NEXT: v_perm_b32 v29, v30, v29, s4
; GFX9-NEXT: v_add_u32_e32 v30, 0x300, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v31, v29, s5
-; GFX9-NEXT: v_add_u32_sdwa v29, v29, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v29, v31, v29, s4
+; GFX9-NEXT: v_add_u32_sdwa v29, v29, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v29, v30, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v31, v30, s5
+; GFX9-NEXT: v_perm_b32 v30, v31, v30, s4
; GFX9-NEXT: v_add_u32_e32 v31, 0x300, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v32, v30, s5
-; GFX9-NEXT: v_add_u32_sdwa v30, v30, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v30, v32, v30, s4
+; GFX9-NEXT: v_add_u32_sdwa v30, v30, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v30, v31, v30 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s5
+; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_add_u32_e32 v32, 0x300, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v33, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v33, v31, s5
-; GFX9-NEXT: v_add_u32_sdwa v31, v31, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v31, v33, v31, s4
+; GFX9-NEXT: v_add_u32_sdwa v31, v31, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v31, v32, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: .LBB39_5: ; %end
; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -67954,15 +68034,15 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v26
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v25
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v24
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v23
@@ -67975,12 +68055,12 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v21
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v20
; SI-NEXT: v_mov_b32_e32 v52, v18
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v19
; SI-NEXT: v_mov_b32_e32 v53, v17
@@ -68020,18 +68100,18 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v42
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v43
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v44
+; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v43
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v44
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v45
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v46
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v47
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v56
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v57
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v59
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v46
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v47
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v56
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v57
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v58
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v59
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v60
; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v61
-; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v62
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v62
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
@@ -68056,7 +68136,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v62
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v63
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v36
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v61
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v50
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -68065,31 +68145,31 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v32
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v59
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v49
; SI-NEXT: v_or_b32_e32 v3, v3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v35
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v48
; SI-NEXT: v_or_b32_e32 v5, v5, v6
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v56
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v39
; SI-NEXT: v_or_b32_e32 v6, v6, v7
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v33
; SI-NEXT: v_or_b32_e32 v7, v7, v8
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v46
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v38
; SI-NEXT: v_or_b32_e32 v8, v8, v9
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v45
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v34
; SI-NEXT: v_or_b32_e32 v9, v9, v10
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v44
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v37
; SI-NEXT: v_or_b32_e32 v10, v10, v11
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v43
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v63
; SI-NEXT: v_or_b32_e32 v11, v11, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v42
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v51
@@ -68147,25 +68227,25 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr42
-; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; kill: killed $vgpr51
; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
@@ -68193,7 +68273,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_or_b32_e32 v18, v18, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: s_waitcnt vmcnt(11)
@@ -68227,7 +68307,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_or_b32_e32 v19, v19, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v20, 0xffff, v20
; SI-NEXT: v_or_b32_e32 v20, v20, v21
@@ -68303,7 +68383,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_cbranch_execz .LBB46_4
; SI-NEXT: ; %bb.3: ; %cmp.true
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, v63
+; SI-NEXT: v_cvt_f32_f16_e32 v0, v36
; SI-NEXT: v_cvt_f32_f16_e32 v2, v50
; SI-NEXT: v_cvt_f32_f16_e32 v1, v62
; SI-NEXT: v_cvt_f32_f16_e32 v3, v61
@@ -68331,7 +68411,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v3, v36
+; SI-NEXT: v_cvt_f32_f16_e32 v3, v49
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_cvt_f32_f16_e32 v6, v57
@@ -68342,7 +68422,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, v49
+; SI-NEXT: v_cvt_f32_f16_e32 v4, v35
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_cvt_f32_f16_e32 v8, v47
@@ -68353,7 +68433,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v5, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v5, v33
+; SI-NEXT: v_cvt_f32_f16_e32 v5, v48
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_cvt_f32_f16_e32 v10, v45
@@ -68364,7 +68444,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, v48
+; SI-NEXT: v_cvt_f32_f16_e32 v6, v39
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_cvt_f32_f16_e32 v12, v43
@@ -68375,7 +68455,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v7, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v7, v35
+; SI-NEXT: v_cvt_f32_f16_e32 v7, v33
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f32_f16_e32 v14, v41
@@ -68386,7 +68466,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v8, v39
+; SI-NEXT: v_cvt_f32_f16_e32 v8, v38
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f32_f16_e32 v16, v55
@@ -68408,23 +68488,23 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v10, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v10, v38
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: v_cvt_f32_f16_e32 v10, v37
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v11, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v11, v37
+; SI-NEXT: v_cvt_f32_f16_e32 v11, v63
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -68553,7 +68633,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_cvt_f32_f16_e32 v25, v25
; SI-NEXT: s_waitcnt vmcnt(2)
@@ -68569,7 +68649,7 @@ define <32 x float> @bitcast_v64f16_to_v32f32(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -88198,341 +88278,366 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: v_readfirstlane_b32 s58, v10
-; SI-NEXT: v_readfirstlane_b32 s59, v9
-; SI-NEXT: v_readfirstlane_b32 s60, v8
-; SI-NEXT: v_readfirstlane_b32 s61, v7
-; SI-NEXT: v_readfirstlane_b32 s76, v0
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:324
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:316
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:300
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:292
-; SI-NEXT: v_readfirstlane_b32 s46, v30
-; SI-NEXT: v_readfirstlane_b32 s6, v29
-; SI-NEXT: v_readfirstlane_b32 s7, v28
-; SI-NEXT: v_readfirstlane_b32 s8, v27
-; SI-NEXT: v_readfirstlane_b32 s9, v26
-; SI-NEXT: v_readfirstlane_b32 s10, v25
-; SI-NEXT: v_readfirstlane_b32 s11, v24
-; SI-NEXT: v_readfirstlane_b32 s12, v23
-; SI-NEXT: v_readfirstlane_b32 s13, v22
-; SI-NEXT: v_readfirstlane_b32 s14, v21
-; SI-NEXT: v_readfirstlane_b32 s15, v20
-; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:16
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:24
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:32
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:144
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:152
+; SI-NEXT: v_readfirstlane_b32 s75, v0
; SI-NEXT: v_readfirstlane_b32 s41, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:332
+; SI-NEXT: v_readfirstlane_b32 s15, v20
; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:16
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:24
-; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:32
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:40
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:96
+; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: v_readfirstlane_b32 s7, v28
+; SI-NEXT: v_readfirstlane_b32 s9, v26
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:192
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:200
; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:208
; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:216
+; SI-NEXT: v_readfirstlane_b32 s76, v30
+; SI-NEXT: v_readfirstlane_b32 s6, v29
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:160
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:168
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:176
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s46, v13
+; SI-NEXT: v_readfirstlane_b32 s47, v12
+; SI-NEXT: v_readfirstlane_b32 s56, v11
+; SI-NEXT: v_readfirstlane_b32 s45, v14
+; SI-NEXT: v_readfirstlane_b32 s43, v16
+; SI-NEXT: v_readfirstlane_b32 s44, v15
+; SI-NEXT: v_readfirstlane_b32 s74, v1
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:60
; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:224
; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:232
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:240
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:144
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:152
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:176
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:184
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:192
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:200
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:208
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:216
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:224
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:232
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:240
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:248
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:256
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:264
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:272
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:280
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:288
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:296
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:304
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:312
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:320
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:328
-; SI-NEXT: v_readfirstlane_b32 s43, v16
-; SI-NEXT: v_readfirstlane_b32 s44, v15
-; SI-NEXT: v_readfirstlane_b32 s45, v14
-; SI-NEXT: v_readfirstlane_b32 s47, v13
-; SI-NEXT: v_readfirstlane_b32 s56, v12
-; SI-NEXT: v_readfirstlane_b32 s57, v11
-; SI-NEXT: v_readfirstlane_b32 s62, v6
-; SI-NEXT: v_readfirstlane_b32 s63, v5
-; SI-NEXT: v_readfirstlane_b32 s72, v4
-; SI-NEXT: v_readfirstlane_b32 s73, v3
-; SI-NEXT: v_readfirstlane_b32 s74, v2
-; SI-NEXT: v_readfirstlane_b32 s75, v1
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:248
+; SI-NEXT: v_readfirstlane_b32 s8, v27
+; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: v_readfirstlane_b32 s11, v24
+; SI-NEXT: v_readfirstlane_b32 s12, v23
+; SI-NEXT: v_readfirstlane_b32 s13, v22
+; SI-NEXT: v_readfirstlane_b32 s14, v21
+; SI-NEXT: v_readfirstlane_b32 s57, v10
+; SI-NEXT: v_readfirstlane_b32 s58, v9
+; SI-NEXT: v_readfirstlane_b32 s59, v8
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_readfirstlane_b32 s62, v5
+; SI-NEXT: v_readfirstlane_b32 s72, v3
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:172
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:180
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:212
+; SI-NEXT: v_readfirstlane_b32 s60, v7
+; SI-NEXT: v_readfirstlane_b32 s63, v4
+; SI-NEXT: v_readfirstlane_b32 s73, v2
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:148
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v16, 8, v33
-; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v34
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v42
-; SI-NEXT: v_lshlrev_b32_e32 v14, 24, v43
-; SI-NEXT: v_lshlrev_b32_e32 v51, 24, v45
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v47
-; SI-NEXT: v_lshlrev_b32_e32 v33, 24, v59
-; SI-NEXT: v_lshlrev_b32_e32 v42, 24, v61
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v20
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v21
-; SI-NEXT: v_lshlrev_b32_e32 v22, 24, v22
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v17
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
+; SI-NEXT: s_and_b64 s[4:5], vcc, exec
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
+; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v37
+; SI-NEXT: v_lshlrev_b32_e32 v50, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v49
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v39
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v38
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v36
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v46
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v33
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v56
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v58
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:84
+; SI-NEXT: v_lshlrev_b32_e32 v45, 8, v34
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v48
+; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 24, v28
+; SI-NEXT: v_lshlrev_b32_e32 v24, 8, v57
+; SI-NEXT: v_lshlrev_b32_e32 v9, 24, v58
+; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v30
+; SI-NEXT: v_lshlrev_b32_e32 v33, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v30, 24, v56
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v59
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v60
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v61
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v62
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v60
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v53
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v62
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:92
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:100
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:108
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v55
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v53
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(14) expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v52
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:132
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v30
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v43
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v27
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v46
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:156
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v49
-; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v18
-; SI-NEXT: v_lshlrev_b32_e32 v17, 8, v36
-; SI-NEXT: v_lshlrev_b32_e32 v18, 24, v41
-; SI-NEXT: v_lshlrev_b32_e32 v44, 24, v57
-; SI-NEXT: v_lshlrev_b32_e32 v55, 24, v24
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v25
-; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:264
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:272
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:280
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:252
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:260
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:276
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v46, 8, v39
-; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v37
-; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v35
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v32
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:284
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:276
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:268
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:260
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:252
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:244
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:228
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:220
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:180
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:172
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:164
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:156
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:220
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:228
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:236
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:148
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:140
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:132
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:20
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:12
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:4
-; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_lshlrev_b32_e32 v31, 8, v31
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:244
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v11
+; SI-NEXT: s_waitcnt vmcnt(11)
+; SI-NEXT: v_lshlrev_b32_e32 v32, 24, v12
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v13
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:288
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:296
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:304
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:312
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:284
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:292
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:308
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v29, 24, v29
+; SI-NEXT: s_waitcnt vmcnt(7)
+; SI-NEXT: v_lshlrev_b32_e32 v12, 8, v12
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v36
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshlrev_b32_e32 v42, 8, v38
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v54
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:320
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:324
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v54
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v46
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:12
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; SI-NEXT: s_cbranch_scc0 .LBB59_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_mov_b32_e32 v40, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; SI-NEXT: s_and_b32 s4, s46, 0xff
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v6, v15, v6
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_or_b32_e32 v38, v38, v11
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v17
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v16
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v38
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v33, v4
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_or_b32_e32 v2, v42, v2
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v35
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_or_b32_e32 v60, v37, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v59
+; SI-NEXT: v_or_b32_e32 v35, v35, v50
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v50, 0xffff, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v58
+; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v35
+; SI-NEXT: v_or_b32_e32 v48, v48, v58
+; SI-NEXT: v_mov_b32_e32 v58, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_and_b32 s4, s76, 0xff
+; SI-NEXT: v_or_b32_e32 v51, s4, v51
+; SI-NEXT: v_and_b32_e32 v51, 0xffff, v51
+; SI-NEXT: v_mov_b32_e32 v62, v32
+; SI-NEXT: v_mov_b32_e32 v32, v13
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v57
+; SI-NEXT: v_mov_b32_e32 v35, v11
+; SI-NEXT: v_or_b32_e32 v11, v51, v60
+; SI-NEXT: v_mov_b32_e32 v59, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v56
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_or_b32_e32 v12, v50, v48
+; SI-NEXT: v_mov_b32_e32 v37, v42
+; SI-NEXT: v_mov_b32_e32 v61, v40
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_and_b32_e32 v18, 0xff, v18
+; SI-NEXT: v_and_b32_e32 v19, 0xff, v19
+; SI-NEXT: v_and_b32_e32 v20, 0xff, v20
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: v_and_b32_e32 v23, 0xff, v23
+; SI-NEXT: v_or_b32_e32 v1, v1, v4
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v20, v0
-; SI-NEXT: s_lshl_b32 s5, s17, 8
-; SI-NEXT: s_lshl_b32 s77, s19, 24
-; SI-NEXT: s_lshl_b32 s78, s23, 24
-; SI-NEXT: s_lshl_b32 s79, s27, 24
-; SI-NEXT: s_lshl_b32 s88, s75, 24
-; SI-NEXT: s_lshl_b32 s89, s63, 24
-; SI-NEXT: s_lshl_b32 s90, s59, 24
-; SI-NEXT: s_lshl_b32 s91, s47, 24
-; SI-NEXT: s_lshl_b32 s92, s42, 24
-; SI-NEXT: s_lshl_b32 s93, s14, 24
-; SI-NEXT: s_lshl_b32 s94, s10, 24
-; SI-NEXT: s_lshl_b32 s95, s6, 24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v7, s4, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_or_b32_e32 v11, v7, v6
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v63
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v7, v18, v7
-; SI-NEXT: v_mov_b32_e32 v6, v13
-; SI-NEXT: v_or_b32_e32 v13, v5, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v62
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v60
-; SI-NEXT: v_or_b32_e32 v5, v5, v19
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v14, v7
-; SI-NEXT: v_mov_b32_e32 v60, v14
-; SI-NEXT: v_or_b32_e32 v14, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v57
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v0, v2, v0
+; SI-NEXT: v_and_b32_e32 v27, 0xff, v27
+; SI-NEXT: v_and_b32_e32 v22, 0xff, v22
+; SI-NEXT: v_or_b32_e32 v23, v26, v23
+; SI-NEXT: v_or_b32_e32 v26, v1, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v55
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v61
+; SI-NEXT: v_or_b32_e32 v27, v27, v58
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
+; SI-NEXT: v_or_b32_e32 v22, v30, v22
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v62, v1
+; SI-NEXT: v_or_b32_e32 v22, v27, v22
+; SI-NEXT: v_or_b32_e32 v27, v0, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v59
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v29, v1
+; SI-NEXT: v_or_b32_e32 v28, v0, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
+; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s5, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s77, s19, 24
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s5, s77, s5
; SI-NEXT: s_or_b32 s4, s4, s5
@@ -88541,6 +88646,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s5, s5, s77
; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s77, s78, s77
; SI-NEXT: s_or_b32 s5, s5, s77
@@ -88549,38 +88655,48 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s26, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
+; SI-NEXT: s_lshl_b32 s79, s27, 24
; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_or_b32 s78, s79, s78
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s28, 0xff
; SI-NEXT: s_lshl_b32 s79, s29, 8
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_mov_b32_e32 v51, v33
+; SI-NEXT: v_or_b32_e32 v13, v13, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s76, 0xff
+; SI-NEXT: s_and_b32 s79, s75, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s78, s78, 0xffff
; SI-NEXT: s_or_b32 s79, s88, s79
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s74, 0xff
-; SI-NEXT: s_lshl_b32 s88, s73, 8
+; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_lshl_b32 s88, s72, 8
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s72, 0xff
+; SI-NEXT: s_and_b32 s88, s63, 0xff
; SI-NEXT: s_lshl_b32 s88, s88, 16
+; SI-NEXT: s_lshl_b32 s89, s62, 24
; SI-NEXT: s_and_b32 s79, s79, 0xffff
; SI-NEXT: s_or_b32 s88, s89, s88
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s62, 0xff
-; SI-NEXT: s_lshl_b32 s89, s61, 8
+; SI-NEXT: s_and_b32 s88, s61, 0xff
+; SI-NEXT: s_lshl_b32 s89, s60, 8
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s60, 0xff
+; SI-NEXT: s_and_b32 s89, s59, 0xff
; SI-NEXT: s_lshl_b32 s89, s89, 16
+; SI-NEXT: s_lshl_b32 s90, s58, 24
; SI-NEXT: s_and_b32 s88, s88, 0xffff
; SI-NEXT: s_or_b32 s89, s90, s89
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s58, 0xff
-; SI-NEXT: s_lshl_b32 s90, s57, 8
+; SI-NEXT: s_and_b32 s89, s57, 0xff
+; SI-NEXT: s_lshl_b32 s90, s56, 8
; SI-NEXT: s_or_b32 s89, s89, s90
-; SI-NEXT: s_and_b32 s90, s56, 0xff
+; SI-NEXT: s_and_b32 s90, s47, 0xff
; SI-NEXT: s_lshl_b32 s90, s90, 16
+; SI-NEXT: s_lshl_b32 s91, s46, 24
; SI-NEXT: s_and_b32 s89, s89, 0xffff
; SI-NEXT: s_or_b32 s90, s91, s90
; SI-NEXT: s_or_b32 s89, s89, s90
@@ -88589,6 +88705,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s90, s90, s91
; SI-NEXT: s_and_b32 s91, s43, 0xff
; SI-NEXT: s_lshl_b32 s91, s91, 16
+; SI-NEXT: s_lshl_b32 s92, s42, 24
; SI-NEXT: s_and_b32 s90, s90, 0xffff
; SI-NEXT: s_or_b32 s91, s92, s91
; SI-NEXT: s_or_b32 s90, s90, s91
@@ -88597,6 +88714,8 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s91, s91, s92
; SI-NEXT: s_and_b32 s92, s15, 0xff
; SI-NEXT: s_lshl_b32 s92, s92, 16
+; SI-NEXT: s_lshl_b32 s93, s14, 24
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
; SI-NEXT: s_and_b32 s91, s91, 0xffff
; SI-NEXT: s_or_b32 s92, s93, s92
; SI-NEXT: s_or_b32 s91, s91, s92
@@ -88605,180 +88724,178 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s11, 0xff
; SI-NEXT: s_lshl_b32 s93, s93, 16
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v51, v7
-; SI-NEXT: v_or_b32_e32 v15, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v32
+; SI-NEXT: s_lshl_b32 s94, s10, 24
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: s_and_b32 s92, s92, 0xffff
; SI-NEXT: s_or_b32 s93, s94, s93
+; SI-NEXT: v_or_b32_e32 v8, v8, v24
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s9, 0xff
; SI-NEXT: s_lshl_b32 s94, s8, 8
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
+; SI-NEXT: v_or_b32_e32 v5, v9, v5
; SI-NEXT: s_or_b32 s93, s93, s94
; SI-NEXT: s_and_b32 s94, s7, 0xff
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v21, 0xff, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xff, v25
+; SI-NEXT: v_or_b32_e32 v24, v8, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v6
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_mov_b32_e32 v50, v33
+; SI-NEXT: v_or_b32_e32 v14, v33, v14
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v13, v13, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v47
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: s_lshl_b32 s94, s94, 16
+; SI-NEXT: s_lshl_b32 s95, s6, 24
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_or_b32_e32 v25, v25, v39
+; SI-NEXT: v_or_b32_e32 v5, v5, v10
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: s_and_b32 s93, s93, 0xffff
; SI-NEXT: s_or_b32 s94, s95, s94
-; SI-NEXT: s_or_b32 s93, s93, s94
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v45
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v48, v7
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_or_b32_e32 v16, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v43
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v52
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v44, v7
-; SI-NEXT: v_or_b32_e32 v17, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v41
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v21, v63, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v25
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v18, v5, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v5, s88
-; SI-NEXT: v_mov_b32_e32 v7, s90
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v19, v3, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v3, v7, v3
+; SI-NEXT: s_or_b32 s93, s93, s94
+; SI-NEXT: v_or_b32_e32 v23, v25, v23
+; SI-NEXT: v_or_b32_e32 v25, v5, v3
+; SI-NEXT: v_mov_b32_e32 v2, s77
; SI-NEXT: v_mov_b32_e32 v3, s78
; SI-NEXT: v_mov_b32_e32 v4, s79
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v20, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v54
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v5, s88
+; SI-NEXT: v_mov_b32_e32 v6, s89
+; SI-NEXT: v_mov_b32_e32 v7, s90
+; SI-NEXT: v_mov_b32_e32 v8, s91
+; SI-NEXT: v_mov_b32_e32 v9, s92
+; SI-NEXT: v_mov_b32_e32 v10, s93
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v21, v1
-; SI-NEXT: v_or_b32_e32 v21, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v50
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v49
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v22, v1
-; SI-NEXT: v_or_b32_e32 v22, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v39
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v23
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v23, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v27
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v48, v33
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v46
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
+; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_or_b32_e32 v33, v42, v33
+; SI-NEXT: v_mov_b32_e32 v46, v44
+; SI-NEXT: v_mov_b32_e32 v44, v42
+; SI-NEXT: v_mov_b32_e32 v42, v38
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_mov_b32_e32 v38, v40
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v40
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v33, v33, v45
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v24
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v55, v1
-; SI-NEXT: v_or_b32_e32 v24, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v59, v1
-; SI-NEXT: v_or_b32_e32 v25, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v29
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v26
-; SI-NEXT: v_or_b32_e32 v0, v0, v31
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v56, v1
-; SI-NEXT: v_or_b32_e32 v26, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v30
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_and_b32_e32 v1, 0xff, v35
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v27, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v61
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v28, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v6
-; SI-NEXT: v_mov_b32_e32 v6, s89
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_or_b32_e32 v1, v43, v1
; SI-NEXT: v_or_b32_e32 v29, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v10
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v0, v0, v46
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v32
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v34
+; SI-NEXT: v_or_b32_e32 v0, v0, v37
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v36, v1
; SI-NEXT: v_or_b32_e32 v30, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v8
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v40
-; SI-NEXT: v_or_b32_e32 v0, v0, v34
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v42
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v52
+; SI-NEXT: v_or_b32_e32 v0, v0, v46
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v37, v1
+; SI-NEXT: v_or_b32_e32 v1, v54, v1
; SI-NEXT: v_or_b32_e32 v31, v0, v1
; SI-NEXT: v_mov_b32_e32 v0, s4
; SI-NEXT: v_mov_b32_e32 v1, s5
-; SI-NEXT: v_mov_b32_e32 v2, s77
-; SI-NEXT: v_mov_b32_e32 v8, s91
-; SI-NEXT: v_mov_b32_e32 v9, s92
-; SI-NEXT: v_mov_b32_e32 v10, s93
; SI-NEXT: s_mov_b64 s[4:5], 0
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v15, v15, v40
+; SI-NEXT: v_or_b32_e32 v15, v33, v15
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v16, v40
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v33
+; SI-NEXT: v_or_b32_e32 v33, v33, v41
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v17, v17, v40
+; SI-NEXT: v_or_b32_e32 v17, v33, v17
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v18
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v18, v18, v40
+; SI-NEXT: v_or_b32_e32 v18, v33, v18
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v19, v19, v40
+; SI-NEXT: v_or_b32_e32 v19, v33, v19
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v20, v20, v40
+; SI-NEXT: v_or_b32_e32 v20, v33, v20
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: v_or_b32_e32 v21, v33, v21
; SI-NEXT: s_branch .LBB59_3
; SI-NEXT: .LBB59_2:
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
-; SI-NEXT: v_mov_b32_e32 v60, v14
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15_vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23_vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31
; SI-NEXT: .LBB59_3: ; %Flow
@@ -88787,9 +88904,9 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB59_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
-; SI-NEXT: s_add_i32 s46, s46, 3
-; SI-NEXT: s_and_b32 s4, s46, 0xff
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; SI-NEXT: s_add_i32 s76, s76, 3
+; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
@@ -88800,14 +88917,14 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s76, s76, 3
-; SI-NEXT: s_add_i32 s74, s74, 3
-; SI-NEXT: s_add_i32 s72, s72, 3
-; SI-NEXT: s_add_i32 s62, s62, 3
-; SI-NEXT: s_add_i32 s60, s60, 3
-; SI-NEXT: s_add_i32 s58, s58, 3
-; SI-NEXT: s_add_i32 s56, s56, 3
-; SI-NEXT: s_and_b32 s22, s56, 0xff
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s63, s63, 3
+; SI-NEXT: s_add_i32 s61, s61, 3
+; SI-NEXT: s_add_i32 s59, s59, 3
+; SI-NEXT: s_add_i32 s57, s57, 3
+; SI-NEXT: s_add_i32 s47, s47, 3
+; SI-NEXT: s_and_b32 s22, s47, 0xff
; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_add_i32 s45, s45, 3
; SI-NEXT: s_add_i32 s43, s43, 3
@@ -88845,22 +88962,22 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
; SI-NEXT: v_mov_b32_e32 v9, s10
; SI-NEXT: v_mov_b32_e32 v10, s6
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, s4, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -88892,38 +89009,38 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s17, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s29, 8
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s19, s76, 0xff
+; SI-NEXT: s_and_b32 s19, s75, 0xff
; SI-NEXT: s_addk_i32 s17, 0x300
-; SI-NEXT: s_lshl_b32 s18, s75, 24
+; SI-NEXT: s_lshl_b32 s18, s74, 24
; SI-NEXT: s_lshl_b32 s19, s19, 16
; SI-NEXT: s_and_b32 s17, s17, 0xffff
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s18, s74, 0xff
-; SI-NEXT: s_lshl_b32 s19, s73, 8
+; SI-NEXT: s_and_b32 s18, s73, 0xff
+; SI-NEXT: s_lshl_b32 s19, s72, 8
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_and_b32 s20, s63, 0xff
; SI-NEXT: s_addk_i32 s18, 0x300
-; SI-NEXT: s_lshl_b32 s19, s63, 24
+; SI-NEXT: s_lshl_b32 s19, s62, 24
; SI-NEXT: s_lshl_b32 s20, s20, 16
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s19, s62, 0xff
-; SI-NEXT: s_lshl_b32 s20, s61, 8
+; SI-NEXT: s_and_b32 s19, s61, 0xff
+; SI-NEXT: s_lshl_b32 s20, s60, 8
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s21, s60, 0xff
+; SI-NEXT: s_and_b32 s21, s59, 0xff
; SI-NEXT: s_addk_i32 s19, 0x300
-; SI-NEXT: s_lshl_b32 s20, s59, 24
+; SI-NEXT: s_lshl_b32 s20, s58, 24
; SI-NEXT: s_lshl_b32 s21, s21, 16
; SI-NEXT: s_and_b32 s19, s19, 0xffff
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s20, s58, 0xff
-; SI-NEXT: s_lshl_b32 s21, s57, 8
+; SI-NEXT: s_and_b32 s20, s57, 0xff
+; SI-NEXT: s_lshl_b32 s21, s56, 8
; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_addk_i32 s20, 0x300
-; SI-NEXT: s_lshl_b32 s21, s47, 24
+; SI-NEXT: s_lshl_b32 s21, s46, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_or_b32 s20, s21, s20
@@ -88943,7 +89060,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_or_b32_e32 v2, v2, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x300, v2
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: s_or_b32 s22, s23, s22
@@ -88971,333 +89088,330 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_or_b32_e32 v0, v3, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; SI-NEXT: v_mov_b32_e32 v3, s17
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v38, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v51, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v63, v0
+; SI-NEXT: v_or_b32_e32 v0, v50, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v62, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v48, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v60, v0
+; SI-NEXT: v_or_b32_e32 v0, v44, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
+; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v38
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v16, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v17, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v18, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v20, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v21, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v22, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v24, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v25, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v26, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v27, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v28, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v29, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -89307,21 +89421,21 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v30, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -89346,7 +89460,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0)
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
;
; VI-LABEL: bitcast_v128i8_to_v16i64_scalar:
@@ -89371,8 +89485,9 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; VI-NEXT: v_readfirstlane_b32 s73, v3
+; VI-NEXT: v_readfirstlane_b32 s74, v2
; VI-NEXT: v_readfirstlane_b32 s47, v13
+; VI-NEXT: v_readfirstlane_b32 s72, v4
; VI-NEXT: v_readfirstlane_b32 s61, v7
; VI-NEXT: v_readfirstlane_b32 s62, v6
; VI-NEXT: v_readfirstlane_b32 s63, v5
@@ -89398,101 +89513,100 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
-; VI-NEXT: v_readfirstlane_b32 s72, v4
-; VI-NEXT: v_readfirstlane_b32 s74, v2
+; VI-NEXT: v_readfirstlane_b32 s73, v3
; VI-NEXT: v_readfirstlane_b32 s75, v1
-; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
+; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:316
+; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
+; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:308
; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:296
; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; VI-NEXT: s_waitcnt vmcnt(13)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: s_waitcnt vmcnt(10)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:320
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:268
+; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:264
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:248
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
-; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:220
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; VI-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; VI-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; VI-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; VI-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:180
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:148
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:136
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -89513,186 +89627,192 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:48
; VI-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; VI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; VI-NEXT: s_cbranch_scc0 .LBB59_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: v_mov_b32_e32 v20, 0xc0c0004
-; VI-NEXT: v_mov_b32_e32 v4, s23
-; VI-NEXT: v_mov_b32_e32 v2, s19
-; VI-NEXT: v_mov_b32_e32 v3, s21
-; VI-NEXT: v_perm_b32 v4, s22, v4, v20
+; VI-NEXT: v_mov_b32_e32 v40, 0xc0c0004
+; VI-NEXT: v_mov_b32_e32 v3, s19
; VI-NEXT: v_mov_b32_e32 v13, s17
-; VI-NEXT: v_perm_b32 v2, s18, v2, v20
-; VI-NEXT: v_perm_b32 v3, s20, v3, v20
-; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_perm_b32 v13, s16, v13, v20
+; VI-NEXT: v_perm_b32 v3, s18, v3, v40
+; VI-NEXT: v_mov_b32_e32 v2, s23
+; VI-NEXT: v_perm_b32 v13, s16, v13, v40
+; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; VI-NEXT: v_mov_b32_e32 v42, s21
+; VI-NEXT: v_perm_b32 v2, s22, v2, v40
+; VI-NEXT: v_perm_b32 v42, s20, v42, v40
; VI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; VI-NEXT: v_mov_b32_e32 v19, v1
-; VI-NEXT: v_or_b32_e32 v1, v3, v4
+; VI-NEXT: v_mov_b32_e32 v20, v0
+; VI-NEXT: v_or_b32_e32 v0, v13, v3
; VI-NEXT: v_mov_b32_e32 v3, s27
-; VI-NEXT: v_mov_b32_e32 v22, v0
-; VI-NEXT: v_or_b32_e32 v0, v13, v2
+; VI-NEXT: v_mov_b32_e32 v19, v1
+; VI-NEXT: v_or_b32_e32 v1, v42, v2
; VI-NEXT: v_mov_b32_e32 v2, s25
-; VI-NEXT: v_perm_b32 v3, s26, v3, v20
-; VI-NEXT: v_perm_b32 v2, s24, v2, v20
+; VI-NEXT: v_perm_b32 v3, s26, v3, v40
+; VI-NEXT: v_mov_b32_e32 v59, v4
+; VI-NEXT: v_perm_b32 v2, s24, v2, v40
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; VI-NEXT: v_mov_b32_e32 v4, s75
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s76, v4, v20
-; VI-NEXT: v_mov_b32_e32 v44, v5
-; VI-NEXT: v_perm_b32 v3, s28, v3, v20
+; VI-NEXT: v_perm_b32 v4, s76, v4, v40
+; VI-NEXT: v_mov_b32_e32 v47, v5
+; VI-NEXT: v_perm_b32 v3, s28, v3, v40
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; VI-NEXT: v_mov_b32_e32 v5, s63
; VI-NEXT: v_or_b32_e32 v3, v3, v4
; VI-NEXT: v_mov_b32_e32 v4, s73
-; VI-NEXT: v_perm_b32 v5, s72, v5, v20
-; VI-NEXT: v_mov_b32_e32 v46, v6
-; VI-NEXT: v_perm_b32 v4, s74, v4, v20
+; VI-NEXT: v_perm_b32 v5, s72, v5, v40
+; VI-NEXT: v_mov_b32_e32 v57, v6
+; VI-NEXT: v_perm_b32 v4, s74, v4, v40
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; VI-NEXT: v_mov_b32_e32 v6, s59
; VI-NEXT: v_or_b32_e32 v4, v4, v5
; VI-NEXT: v_mov_b32_e32 v5, s61
-; VI-NEXT: v_perm_b32 v6, s60, v6, v20
-; VI-NEXT: v_mov_b32_e32 v45, v7
-; VI-NEXT: v_perm_b32 v5, s62, v5, v20
+; VI-NEXT: v_perm_b32 v6, s60, v6, v40
+; VI-NEXT: v_mov_b32_e32 v56, v7
+; VI-NEXT: v_perm_b32 v5, s62, v5, v40
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; VI-NEXT: v_mov_b32_e32 v7, s47
; VI-NEXT: v_or_b32_e32 v5, v5, v6
; VI-NEXT: v_mov_b32_e32 v6, s57
-; VI-NEXT: v_perm_b32 v7, s56, v7, v20
-; VI-NEXT: v_mov_b32_e32 v47, v40
-; VI-NEXT: v_mov_b32_e32 v40, v8
-; VI-NEXT: v_perm_b32 v6, s58, v6, v20
+; VI-NEXT: v_perm_b32 v7, s56, v7, v40
+; VI-NEXT: v_mov_b32_e32 v43, v8
+; VI-NEXT: v_perm_b32 v6, s58, v6, v40
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_mov_b32_e32 v7, s45
-; VI-NEXT: v_perm_b32 v8, s44, v8, v20
-; VI-NEXT: v_mov_b32_e32 v43, v9
-; VI-NEXT: v_perm_b32 v7, s46, v7, v20
+; VI-NEXT: v_perm_b32 v8, s44, v8, v40
+; VI-NEXT: v_mov_b32_e32 v46, v9
+; VI-NEXT: v_perm_b32 v7, s46, v7, v40
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s15
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s41
-; VI-NEXT: v_perm_b32 v9, s40, v9, v20
-; VI-NEXT: v_mov_b32_e32 v56, v41
-; VI-NEXT: v_mov_b32_e32 v41, v10
-; VI-NEXT: v_perm_b32 v8, s42, v8, v20
+; VI-NEXT: v_perm_b32 v9, s40, v9, v40
+; VI-NEXT: v_mov_b32_e32 v44, v10
+; VI-NEXT: v_perm_b32 v8, s42, v8, v40
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_or_b32_e32 v8, v8, v9
; VI-NEXT: v_mov_b32_e32 v9, s13
-; VI-NEXT: v_perm_b32 v10, s12, v10, v20
-; VI-NEXT: v_mov_b32_e32 v42, v11
-; VI-NEXT: v_perm_b32 v9, s14, v9, v20
+; VI-NEXT: v_perm_b32 v10, s12, v10, v40
+; VI-NEXT: v_mov_b32_e32 v58, v41
+; VI-NEXT: v_mov_b32_e32 v41, v11
+; VI-NEXT: v_perm_b32 v9, s14, v9, v40
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_or_b32_e32 v9, v9, v10
; VI-NEXT: v_mov_b32_e32 v10, s9
-; VI-NEXT: v_perm_b32 v11, s8, v11, v20
-; VI-NEXT: v_perm_b32 v10, s10, v10, v20
+; VI-NEXT: v_perm_b32 v11, s8, v11, v40
+; VI-NEXT: v_perm_b32 v10, s10, v10, v40
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
+; VI-NEXT: v_perm_b32 v18, v41, v18, s4
+; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; VI-NEXT: v_perm_b32 v19, v63, v19, s4
+; VI-NEXT: v_or_b32_e32 v18, v19, v18
+; VI-NEXT: v_perm_b32 v19, v22, v45, s4
+; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; VI-NEXT: v_perm_b32 v20, v62, v20, s4
+; VI-NEXT: v_or_b32_e32 v19, v20, v19
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: v_perm_b32 v12, v12, v61, s4
; VI-NEXT: v_perm_b32 v14, v52, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_perm_b32 v15, v48, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_perm_b32 v16, v50, v16, s4
-; VI-NEXT: v_perm_b32 v18, v42, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_perm_b32 v17, v35, v17, s4
-; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
-; VI-NEXT: v_perm_b32 v19, v63, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
-; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_perm_b32 v19, v23, v24, s4
-; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; VI-NEXT: v_perm_b32 v23, v34, v33, s4
-; VI-NEXT: v_perm_b32 v24, v41, v43, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v11, s6, v11, v20
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v22, v30, v31, s4
+; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: v_perm_b32 v11, s6, v11, v40
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: s_waitcnt vmcnt(2)
+; VI-NEXT: v_perm_b32 v20, v25, v20, s4
+; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v25, v56, v57, s4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v20, v13, s4
+; VI-NEXT: v_perm_b32 v13, v40, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_e32 v11, v11, v13
; VI-NEXT: v_perm_b32 v13, v21, v60, s4
@@ -89700,88 +89820,84 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v27, v29, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; VI-NEXT: v_perm_b32 v20, v57, v28, s4
-; VI-NEXT: v_or_b32_e32 v13, v20, v13
-; VI-NEXT: v_perm_b32 v20, v55, v51, s4
-; VI-NEXT: v_or_b32_e32 v14, v20, v14
-; VI-NEXT: v_perm_b32 v20, v54, v53, s4
-; VI-NEXT: v_or_b32_e32 v15, v20, v15
-; VI-NEXT: v_perm_b32 v20, v39, v36, s4
-; VI-NEXT: v_or_b32_e32 v16, v20, v16
-; VI-NEXT: v_perm_b32 v20, v49, v38, s4
-; VI-NEXT: v_or_b32_e32 v17, v20, v17
-; VI-NEXT: v_perm_b32 v20, v62, v22, s4
-; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
-; VI-NEXT: v_perm_b32 v21, v25, v58, s4
-; VI-NEXT: v_perm_b32 v22, v30, v31, s4
-; VI-NEXT: v_perm_b32 v25, v45, v46, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v26, v20, s4
-; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v21, v26, v28, s4
+; VI-NEXT: v_or_b32_e32 v13, v21, v13
+; VI-NEXT: v_perm_b32 v21, v55, v32, s4
+; VI-NEXT: v_or_b32_e32 v14, v21, v14
+; VI-NEXT: v_perm_b32 v21, v54, v53, s4
+; VI-NEXT: v_or_b32_e32 v15, v21, v15
+; VI-NEXT: v_perm_b32 v21, v39, v36, s4
+; VI-NEXT: v_or_b32_e32 v16, v21, v16
+; VI-NEXT: v_perm_b32 v21, v49, v38, s4
+; VI-NEXT: v_or_b32_e32 v17, v21, v17
+; VI-NEXT: v_perm_b32 v21, v23, v24, s4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v23, v34, v33, s4
+; VI-NEXT: v_perm_b32 v24, v44, v46, s4
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v37, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v32, v22, s4
+; VI-NEXT: v_perm_b32 v22, v51, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v40, v23, s4
+; VI-NEXT: v_perm_b32 v23, v43, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v44, v24, s4
+; VI-NEXT: v_perm_b32 v24, v47, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v25, v26, s4
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v25
+; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v25
; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v26, v25, s4
-; VI-NEXT: v_or_b32_e32 v25, v25, v27
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v25, v59, v25, s4
+; VI-NEXT: v_or_b32_e32 v25, v25, v26
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v26, v26, v27, s4
+; VI-NEXT: v_perm_b32 v26, v26, v58, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_or_b32_e32 v26, v26, v28
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v27, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; VI-NEXT: v_or_b32_e32 v27, v27, v29
-; VI-NEXT: v_perm_b32 v29, v47, v56, s4
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v28, v59, v28, s4
+; VI-NEXT: v_perm_b32 v28, v29, v28, s4
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v28, v29, v28
; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
-; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v30, v31, v30, s4
@@ -89796,16 +89912,16 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v31, v32, v31, s4
; VI-NEXT: v_or_b32_e32 v30, v31, v30
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v31, v32, v31, s4
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v31, v31, v32, s4
+; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v31
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v32, v33, v32, s4
-; VI-NEXT: v_or_b32_e32 v31, v32, v31
+; VI-NEXT: v_perm_b32 v31, v33, v31, s4
+; VI-NEXT: v_or_b32_e32 v31, v31, v32
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB59_3
; VI-NEXT: .LBB59_2:
@@ -89917,7 +90033,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x300, v10
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_add_u32_e32 v0, vcc, 0x3000000, v0
@@ -89932,12 +90048,12 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v9, vcc, 0x3000000, v9
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x3000000, v10
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v12, s6, v11, v12
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v12
+; VI-NEXT: v_perm_b32 v11, s6, v11, v12
+; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v11
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v11, v13, v11, s4
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
@@ -89948,57 +90064,57 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v12
; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v13, v12, s4
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
+; VI-NEXT: v_perm_b32 v12, v13, v12, s4
+; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v12
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v12, v14, v12, s4
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v12, v12, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v12, vcc, 0x3000000, v12
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v14, v14, v13, s4
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
+; VI-NEXT: v_perm_b32 v13, v14, v13, s4
+; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v13
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v15, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x3000000, v13
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v15, v15, v14, s4
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v15
+; VI-NEXT: v_perm_b32 v14, v15, v14, s4
+; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v14
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v14, v16, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v14, v14, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x3000000, v14
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v15
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v16, v16, v15, s4
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v16
+; VI-NEXT: v_perm_b32 v15, v16, v15, s4
+; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v15
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v15
; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
@@ -90006,151 +90122,151 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_perm_b32 v15, v17, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_or_b32_sdwa v15, v15, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v15, vcc, 0x3000000, v15
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v17, v17, v16, s4
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v17
+; VI-NEXT: v_perm_b32 v16, v17, v16, s4
+; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v16
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v16, v18, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_or_b32_sdwa v16, v16, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v16, vcc, 0x3000000, v16
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v18, v18, v17, s4
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v18
+; VI-NEXT: v_perm_b32 v17, v18, v17, s4
+; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v17
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v17, v19, v17, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; VI-NEXT: v_or_b32_sdwa v17, v17, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v17, vcc, 0x3000000, v17
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v19, v19, v18, s4
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v19
+; VI-NEXT: v_perm_b32 v18, v19, v18, s4
+; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v18
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v18, v20, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; VI-NEXT: v_or_b32_sdwa v18, v18, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v18, vcc, 0x3000000, v18
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v20, v19, s4
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v20
+; VI-NEXT: v_perm_b32 v19, v20, v19, s4
+; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v19
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v19, v21, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; VI-NEXT: v_or_b32_sdwa v19, v19, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v19, vcc, 0x3000000, v19
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v21, v21, v20, s4
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v21
+; VI-NEXT: v_perm_b32 v20, v21, v20, s4
+; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v20
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v20, v22, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; VI-NEXT: v_or_b32_sdwa v20, v20, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v20, vcc, 0x3000000, v20
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v22, v21, s4
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v22
+; VI-NEXT: v_perm_b32 v21, v22, v21, s4
+; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v21
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v23, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_sdwa v21, v21, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v21, vcc, 0x3000000, v21
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v23, v22, s4
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v23
+; VI-NEXT: v_perm_b32 v22, v23, v22, s4
+; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v22
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v22, v24, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_sdwa v22, v22, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v22, vcc, 0x3000000, v22
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v24, v23, s4
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v24
+; VI-NEXT: v_perm_b32 v23, v24, v23, s4
+; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v23
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v23, v25, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_sdwa v23, v23, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v23, vcc, 0x3000000, v23
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v25, v24, s4
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v25
+; VI-NEXT: v_perm_b32 v24, v25, v24, s4
+; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v24
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v24, v26, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_sdwa v24, v24, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v24, vcc, 0x3000000, v24
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v25
@@ -90158,55 +90274,55 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v26, v25, s4
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x300, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v27, v25, s4
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_or_b32_sdwa v25, v25, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v25, vcc, 0x3000000, v25
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x300, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; VI-NEXT: v_or_b32_sdwa v26, v26, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x3000000, v26
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x300, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v29, v27, s4
; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; VI-NEXT: v_or_b32_sdwa v27, v27, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x3000000, v27
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v28, v29, v28, s4
; VI-NEXT: v_add_u32_e32 v29, vcc, 0x300, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v28
; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
@@ -90214,11 +90330,11 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; VI-NEXT: v_perm_b32 v28, v30, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; VI-NEXT: v_or_b32_sdwa v28, v28, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x3000000, v28
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v29
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_add_u32_e32 v30, vcc, 0x300, v29
@@ -90305,8 +90421,9 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; GFX9-NEXT: v_readfirstlane_b32 s73, v3
-; GFX9-NEXT: v_readfirstlane_b32 s47, v13
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s72, v4
+; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s61, v7
; GFX9-NEXT: v_readfirstlane_b32 s62, v6
; GFX9-NEXT: v_readfirstlane_b32 s63, v5
@@ -90323,31 +90440,28 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s13, v23
; GFX9-NEXT: v_readfirstlane_b32 s14, v22
; GFX9-NEXT: v_readfirstlane_b32 s15, v21
-; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s41, v19
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
+; GFX9-NEXT: v_readfirstlane_b32 s47, v13
; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
-; GFX9-NEXT: v_readfirstlane_b32 s72, v4
-; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s73, v3
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
-; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
+; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:320
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:316
+; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:296
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; GFX9-NEXT: s_waitcnt vmcnt(13)
+; GFX9-NEXT: s_waitcnt vmcnt(11)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -90357,82 +90471,86 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; GFX9-NEXT: s_nop 0
+; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:264
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:248
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
+; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; GFX9-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; GFX9-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; GFX9-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
-; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
+; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:176
; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; GFX9-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:148
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:136
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -90440,108 +90558,109 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:108
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:104
-; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:100
; GFX9-NEXT: buffer_load_ushort v38, off, s[0:3], s32 offset:96
-; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v50, off, s[0:3], s32 offset:80
; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v15, off, s[0:3], s32 offset:72
-; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:68
-; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:64
-; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
-; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
+; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:68
+; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:64
+; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:60
+; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:56
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(55)
-; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(54)
-; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(49)
-; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(62)
; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
@@ -90561,7 +90680,8 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_mov_b32_e32 v19, v1
; GFX9-NEXT: v_or_b32_e32 v1, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v3, s27
-; GFX9-NEXT: v_mov_b32_e32 v22, v0
+; GFX9-NEXT: v_mov_b32_e32 v46, v42
+; GFX9-NEXT: v_mov_b32_e32 v42, v0
; GFX9-NEXT: v_or_b32_e32 v0, v13, v2
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v20
@@ -90571,28 +90691,28 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_or_b32_e32 v2, v2, v3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v20
-; GFX9-NEXT: v_mov_b32_e32 v44, v5
+; GFX9-NEXT: v_mov_b32_e32 v45, v5
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s63
; GFX9-NEXT: v_or_b32_e32 v3, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v4, s73
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v20
-; GFX9-NEXT: v_mov_b32_e32 v46, v6
+; GFX9-NEXT: v_mov_b32_e32 v56, v6
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s59
; GFX9-NEXT: v_or_b32_e32 v4, v4, v5
; GFX9-NEXT: v_mov_b32_e32 v5, s61
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v20
-; GFX9-NEXT: v_mov_b32_e32 v45, v7
+; GFX9-NEXT: v_mov_b32_e32 v47, v7
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s47
; GFX9-NEXT: v_or_b32_e32 v5, v5, v6
; GFX9-NEXT: v_mov_b32_e32 v6, s57
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v20
-; GFX9-NEXT: v_mov_b32_e32 v47, v40
+; GFX9-NEXT: v_mov_b32_e32 v57, v40
; GFX9-NEXT: v_mov_b32_e32 v40, v8
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
@@ -90600,22 +90720,21 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_or_b32_e32 v6, v6, v7
; GFX9-NEXT: v_mov_b32_e32 v7, s45
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v20
-; GFX9-NEXT: v_mov_b32_e32 v43, v9
+; GFX9-NEXT: v_mov_b32_e32 v44, v9
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v9, s15
; GFX9-NEXT: v_or_b32_e32 v7, v7, v8
; GFX9-NEXT: v_mov_b32_e32 v8, s41
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v20
-; GFX9-NEXT: v_mov_b32_e32 v56, v41
-; GFX9-NEXT: v_mov_b32_e32 v41, v10
+; GFX9-NEXT: v_mov_b32_e32 v43, v10
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; GFX9-NEXT: v_mov_b32_e32 v10, s11
; GFX9-NEXT: v_or_b32_e32 v8, v8, v9
; GFX9-NEXT: v_mov_b32_e32 v9, s13
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v20
-; GFX9-NEXT: v_mov_b32_e32 v42, v11
+; GFX9-NEXT: v_mov_b32_e32 v25, v11
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: v_mov_b32_e32 v11, s7
@@ -90628,22 +90747,21 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v12, v12, v61, s4
-; GFX9-NEXT: v_perm_b32 v14, v52, v14, s4
-; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v32, v15, s4
+; GFX9-NEXT: v_perm_b32 v14, v33, v14, s4
+; GFX9-NEXT: v_perm_b32 v15, v39, v15, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; GFX9-NEXT: v_perm_b32 v16, v53, v16, s4
-; GFX9-NEXT: v_perm_b32 v18, v42, v18, s4
+; GFX9-NEXT: v_perm_b32 v18, v25, v18, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; GFX9-NEXT: v_perm_b32 v17, v35, v17, s4
+; GFX9-NEXT: v_perm_b32 v17, v49, v17, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; GFX9-NEXT: v_perm_b32 v19, v63, v19, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; GFX9-NEXT: v_or_b32_e32 v18, v19, v18
-; GFX9-NEXT: v_perm_b32 v19, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v19, v22, v41, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; GFX9-NEXT: v_perm_b32 v23, v34, v33, s4
-; GFX9-NEXT: v_perm_b32 v24, v41, v43, s4
+; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
+; GFX9-NEXT: v_perm_b32 v25, v47, v56, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v11, s6, v11, v20
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
@@ -90657,81 +90775,83 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_or_b32_e32 v12, v12, v13
; GFX9-NEXT: v_perm_b32 v13, v27, v29, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; GFX9-NEXT: v_perm_b32 v20, v57, v28, s4
+; GFX9-NEXT: v_perm_b32 v20, v26, v28, s4
; GFX9-NEXT: v_or_b32_e32 v13, v20, v13
-; GFX9-NEXT: v_perm_b32 v20, v55, v48, s4
-; GFX9-NEXT: v_or_b32_e32 v14, v20, v14
-; GFX9-NEXT: v_perm_b32 v20, v54, v39, s4
+; GFX9-NEXT: v_perm_b32 v20, v52, v48, s4
+; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; GFX9-NEXT: v_or_b32_e32 v14, v14, v20
+; GFX9-NEXT: v_perm_b32 v20, v32, v54, s4
; GFX9-NEXT: v_or_b32_e32 v15, v20, v15
; GFX9-NEXT: v_perm_b32 v20, v36, v50, s4
; GFX9-NEXT: v_or_b32_e32 v16, v20, v16
-; GFX9-NEXT: v_perm_b32 v20, v49, v38, s4
+; GFX9-NEXT: v_perm_b32 v20, v35, v38, s4
; GFX9-NEXT: v_or_b32_e32 v17, v20, v17
-; GFX9-NEXT: v_perm_b32 v20, v62, v22, s4
+; GFX9-NEXT: v_perm_b32 v20, v62, v42, s4
; GFX9-NEXT: v_or_b32_e32 v19, v20, v19
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v21, v25, v58, s4
-; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
-; GFX9-NEXT: v_perm_b32 v25, v45, v46, s4
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v21, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v23, v34, v55, s4
+; GFX9-NEXT: v_perm_b32 v24, v43, v44, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v26, v20, s4
+; GFX9-NEXT: v_perm_b32 v20, v58, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; GFX9-NEXT: v_or_b32_e32 v20, v21, v20
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v21, v37, v21, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; GFX9-NEXT: v_or_b32_e32 v21, v22, v21
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v22, v51, v22, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; GFX9-NEXT: v_or_b32_e32 v22, v23, v22
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v23, v40, v23, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; GFX9-NEXT: v_or_b32_e32 v23, v24, v23
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v44, v24, s4
+; GFX9-NEXT: v_perm_b32 v24, v45, v24, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; GFX9-NEXT: v_or_b32_e32 v24, v25, v24
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v26, s4
+; GFX9-NEXT: v_perm_b32 v25, v25, v46, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_or_b32_e32 v25, v25, v27
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v26, v27, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_or_b32_e32 v26, v26, v28
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v27, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: v_or_b32_e32 v27, v27, v29
-; GFX9-NEXT: v_perm_b32 v29, v47, v56, s4
-; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
; GFX9-NEXT: v_perm_b32 v28, v59, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_perm_b32 v29, v57, v29, s4
; GFX9-NEXT: v_or_b32_e32 v28, v29, v28
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
@@ -90753,10 +90873,10 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_or_b32_e32 v30, v31, v30
-; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
+; GFX9-NEXT: v_perm_b32 v31, v31, v32, s4
; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -90779,21 +90899,21 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_mov_b32_e32 v12, 0xc0c0004
; GFX9-NEXT: s_add_i32 s18, s18, 3
; GFX9-NEXT: v_mov_b32_e32 v1, s19
-; GFX9-NEXT: s_add_i32 s20, s20, 3
-; GFX9-NEXT: v_mov_b32_e32 v2, s21
-; GFX9-NEXT: s_add_i32 s22, s22, 3
-; GFX9-NEXT: v_mov_b32_e32 v3, s23
; GFX9-NEXT: v_perm_b32 v0, s16, v0, v12
-; GFX9-NEXT: s_movk_i32 s4, 0x300
+; GFX9-NEXT: s_movk_i32 s5, 0x300
; GFX9-NEXT: v_perm_b32 v1, s18, v1, v12
-; GFX9-NEXT: v_perm_b32 v2, s20, v2, v12
-; GFX9-NEXT: v_perm_b32 v3, s22, v3, v12
; GFX9-NEXT: v_add_u32_e32 v0, 0x300, v0
-; GFX9-NEXT: v_add_u32_sdwa v1, v1, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v1, v1, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_or_b32_sdwa v1, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX9-NEXT: s_add_i32 s20, s20, 3
+; GFX9-NEXT: v_mov_b32_e32 v1, s21
+; GFX9-NEXT: s_add_i32 s22, s22, 3
+; GFX9-NEXT: v_mov_b32_e32 v2, s23
+; GFX9-NEXT: v_perm_b32 v1, s20, v1, v12
+; GFX9-NEXT: v_perm_b32 v2, s22, v2, v12
+; GFX9-NEXT: v_add_u32_e32 v1, 0x300, v1
+; GFX9-NEXT: v_add_u32_sdwa v2, v2, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s24, s24, 3
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: s_add_i32 s26, s26, 3
@@ -90801,7 +90921,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v12
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v12
; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v3, v3, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v2, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s28, s28, 3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
@@ -90810,7 +90930,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v12
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v12
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
-; GFX9-NEXT: v_add_u32_sdwa v4, v4, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v3, v3, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_mov_b32_e32 v4, s73
@@ -90819,7 +90939,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v12
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v12
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
-; GFX9-NEXT: v_add_u32_sdwa v5, v5, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v4, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s62, s62, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s61
@@ -90828,7 +90948,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v12
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v12
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u32_sdwa v6, v6, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v5, v5, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s58, s58, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s57
@@ -90837,7 +90957,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v12
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v12
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
-; GFX9-NEXT: v_add_u32_sdwa v7, v7, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v6, v6, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s46, s46, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s45
@@ -90846,7 +90966,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v12
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v12
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
-; GFX9-NEXT: v_add_u32_sdwa v8, v8, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v8, v8, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v7, v7, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v8, s41
@@ -90855,7 +90975,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v12
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v12
; GFX9-NEXT: v_add_u32_e32 v8, 0x300, v8
-; GFX9-NEXT: v_add_u32_sdwa v9, v9, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v9, v9, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v8, v8, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s14, s14, 3
; GFX9-NEXT: v_mov_b32_e32 v9, s13
@@ -90864,7 +90984,7 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v12
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v12
; GFX9-NEXT: v_add_u32_e32 v9, 0x300, v9
-; GFX9-NEXT: v_add_u32_sdwa v10, v10, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v10, v10, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v9, v9, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s10, s10, 3
; GFX9-NEXT: v_mov_b32_e32 v10, s9
@@ -90873,321 +90993,321 @@ define inreg <16 x i64> @bitcast_v128i8_to_v16i64_scalar(<128 x i8> inreg %a, i3
; GFX9-NEXT: v_perm_b32 v10, s10, v10, v12
; GFX9-NEXT: v_perm_b32 v11, s8, v11, v12
; GFX9-NEXT: v_add_u32_e32 v10, 0x300, v10
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_add_i32 s6, s6, 3
-; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
+; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, s6, v11, v12
+; GFX9-NEXT: v_perm_b32 v11, s6, v11, v12
+; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v12
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v11, v13, v11, s5
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v11, v13, v11, s4
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v12
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v13, v12, s5
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
+; GFX9-NEXT: v_perm_b32 v12, v13, v12, s4
+; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v12
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, v14, v12, s5
-; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v12, v14, v12, s4
+; GFX9-NEXT: v_add_u32_sdwa v12, v12, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v12, v13, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v14, v13, s5
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
+; GFX9-NEXT: v_perm_b32 v13, v14, v13, s4
+; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v13
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v15, v13, s5
-; GFX9-NEXT: v_add_u32_sdwa v13, v13, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v13, v15, v13, s4
+; GFX9-NEXT: v_add_u32_sdwa v13, v13, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v15, v14, s5
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v15
+; GFX9-NEXT: v_perm_b32 v14, v15, v14, s4
+; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v14
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v16, v14, s5
-; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v16, v14, s4
+; GFX9-NEXT: v_add_u32_sdwa v14, v14, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v16, v15, s5
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v16
+; GFX9-NEXT: v_perm_b32 v15, v16, v15, s4
+; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v15
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v17, v15, s5
-; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v15, v17, v15, s4
+; GFX9-NEXT: v_add_u32_sdwa v15, v15, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v15, v16, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v17, v16, s5
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v17
+; GFX9-NEXT: v_perm_b32 v16, v17, v16, s4
+; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v16
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v18, v16, s5
-; GFX9-NEXT: v_add_u32_sdwa v16, v16, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v16, v18, v16, s4
+; GFX9-NEXT: v_add_u32_sdwa v16, v16, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v16, v17, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v18, v17, s5
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v18
+; GFX9-NEXT: v_perm_b32 v17, v18, v17, s4
+; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v17
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v19, v17, s5
-; GFX9-NEXT: v_add_u32_sdwa v17, v17, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v17, v19, v17, s4
+; GFX9-NEXT: v_add_u32_sdwa v17, v17, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v17, v18, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v19, v18, s5
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v19
+; GFX9-NEXT: v_perm_b32 v18, v19, v18, s4
+; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v18
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v20, v18, s5
-; GFX9-NEXT: v_add_u32_sdwa v18, v18, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v18, v20, v18, s4
+; GFX9-NEXT: v_add_u32_sdwa v18, v18, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v18, v19, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v20, v19, s5
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v20
+; GFX9-NEXT: v_perm_b32 v19, v20, v19, s4
+; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v19
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v21, v19, s5
-; GFX9-NEXT: v_add_u32_sdwa v19, v19, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v19, v21, v19, s4
+; GFX9-NEXT: v_add_u32_sdwa v19, v19, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v19, v20, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v21, v20, s5
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v21
+; GFX9-NEXT: v_perm_b32 v20, v21, v20, s4
+; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v20
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v22, v20, s5
-; GFX9-NEXT: v_add_u32_sdwa v20, v20, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v20, v22, v20, s4
+; GFX9-NEXT: v_add_u32_sdwa v20, v20, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v20, v21, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v22, v21, s5
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v22
+; GFX9-NEXT: v_perm_b32 v21, v22, v21, s4
+; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v21
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v23, v21, s5
-; GFX9-NEXT: v_add_u32_sdwa v21, v21, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v21, v23, v21, s4
+; GFX9-NEXT: v_add_u32_sdwa v21, v21, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v21, v22, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v23, v22, s5
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v23
+; GFX9-NEXT: v_perm_b32 v22, v23, v22, s4
+; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v22
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v24, v22, s5
-; GFX9-NEXT: v_add_u32_sdwa v22, v22, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v22, v24, v22, s4
+; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v22, v23, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v24, v23, s5
+; GFX9-NEXT: v_perm_b32 v23, v24, v23, s4
+; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v23
; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v24
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v25, v23, s5
-; GFX9-NEXT: v_add_u32_sdwa v23, v23, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v23, v25, v23, s4
+; GFX9-NEXT: v_add_u32_sdwa v23, v23, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v23, v24, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v24, s5
+; GFX9-NEXT: v_perm_b32 v24, v25, v24, s4
+; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v24
; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v25
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v26, v24, s5
-; GFX9-NEXT: v_add_u32_sdwa v24, v24, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v24, v26, v24, s4
+; GFX9-NEXT: v_add_u32_sdwa v24, v24, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v24, v25, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v26, v25, s5
+; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_add_u32_e32 v26, 0x300, v25
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v27, v25, s5
-; GFX9-NEXT: v_add_u32_sdwa v25, v25, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v25, v27, v25, s4
+; GFX9-NEXT: v_add_u32_sdwa v25, v25, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v25, v26, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v27, v26, s5
+; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_add_u32_e32 v27, 0x300, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v28, v26, s5
-; GFX9-NEXT: v_add_u32_sdwa v26, v26, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v26, v28, v26, s4
+; GFX9-NEXT: v_add_u32_sdwa v26, v26, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v26, v27, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v28, v27, s5
+; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: v_add_u32_e32 v28, 0x300, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v29, v27, s5
-; GFX9-NEXT: v_add_u32_sdwa v27, v27, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v27, v29, v27, s4
+; GFX9-NEXT: v_add_u32_sdwa v27, v27, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v27, v28, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v28
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v29, v28, s5
+; GFX9-NEXT: v_perm_b32 v28, v29, v28, s4
; GFX9-NEXT: v_add_u32_e32 v29, 0x300, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v30, v28, s5
-; GFX9-NEXT: v_add_u32_sdwa v28, v28, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v28, v30, v28, s4
+; GFX9-NEXT: v_add_u32_sdwa v28, v28, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v28, v29, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v30, v29, s5
+; GFX9-NEXT: v_perm_b32 v29, v30, v29, s4
; GFX9-NEXT: v_add_u32_e32 v30, 0x300, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v31, v29, s5
-; GFX9-NEXT: v_add_u32_sdwa v29, v29, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v29, v31, v29, s4
+; GFX9-NEXT: v_add_u32_sdwa v29, v29, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v29, v30, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v31, v30, s5
+; GFX9-NEXT: v_perm_b32 v30, v31, v30, s4
; GFX9-NEXT: v_add_u32_e32 v31, 0x300, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v32, v30, s5
-; GFX9-NEXT: v_add_u32_sdwa v30, v30, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v30, v32, v30, s4
+; GFX9-NEXT: v_add_u32_sdwa v30, v30, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v30, v31, v30 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s5
+; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_add_u32_e32 v32, 0x300, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v33, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v33, v31, s5
-; GFX9-NEXT: v_add_u32_sdwa v31, v31, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v31, v33, v31, s4
+; GFX9-NEXT: v_add_u32_sdwa v31, v31, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v31, v32, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: .LBB59_5: ; %end
; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -102056,15 +102176,15 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v26
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v25
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v24
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v23
@@ -102077,12 +102197,12 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v21
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v20
; SI-NEXT: v_mov_b32_e32 v52, v18
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v19
; SI-NEXT: v_mov_b32_e32 v53, v17
@@ -102122,18 +102242,18 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v42
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v43
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v44
+; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v43
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v44
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v45
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v46
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v47
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v56
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v57
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v59
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v46
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v47
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v56
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v57
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v58
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v59
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v60
; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v61
-; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v62
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v62
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
@@ -102158,7 +102278,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v62
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v63
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v36
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v61
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v50
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -102167,31 +102287,31 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v32
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v59
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v49
; SI-NEXT: v_or_b32_e32 v3, v3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v35
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v48
; SI-NEXT: v_or_b32_e32 v5, v5, v6
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v56
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v39
; SI-NEXT: v_or_b32_e32 v6, v6, v7
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v33
; SI-NEXT: v_or_b32_e32 v7, v7, v8
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v46
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v38
; SI-NEXT: v_or_b32_e32 v8, v8, v9
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v45
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v34
; SI-NEXT: v_or_b32_e32 v9, v9, v10
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v44
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v37
; SI-NEXT: v_or_b32_e32 v10, v10, v11
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v43
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v63
; SI-NEXT: v_or_b32_e32 v11, v11, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v42
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v51
@@ -102249,25 +102369,25 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr42
-; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; kill: killed $vgpr51
; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
@@ -102295,7 +102415,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_or_b32_e32 v18, v18, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: s_waitcnt vmcnt(11)
@@ -102329,7 +102449,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_or_b32_e32 v19, v19, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v20, 0xffff, v20
; SI-NEXT: v_or_b32_e32 v20, v20, v21
@@ -102405,7 +102525,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_cbranch_execz .LBB66_4
; SI-NEXT: ; %bb.3: ; %cmp.true
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, v63
+; SI-NEXT: v_cvt_f32_f16_e32 v0, v36
; SI-NEXT: v_cvt_f32_f16_e32 v2, v50
; SI-NEXT: v_cvt_f32_f16_e32 v1, v62
; SI-NEXT: v_cvt_f32_f16_e32 v3, v61
@@ -102433,7 +102553,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v3, v36
+; SI-NEXT: v_cvt_f32_f16_e32 v3, v49
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_cvt_f32_f16_e32 v6, v57
@@ -102444,7 +102564,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, v49
+; SI-NEXT: v_cvt_f32_f16_e32 v4, v35
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_cvt_f32_f16_e32 v8, v47
@@ -102455,7 +102575,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v5, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v5, v33
+; SI-NEXT: v_cvt_f32_f16_e32 v5, v48
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_cvt_f32_f16_e32 v10, v45
@@ -102466,7 +102586,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, v48
+; SI-NEXT: v_cvt_f32_f16_e32 v6, v39
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_cvt_f32_f16_e32 v12, v43
@@ -102477,7 +102597,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v7, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v7, v35
+; SI-NEXT: v_cvt_f32_f16_e32 v7, v33
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f32_f16_e32 v14, v41
@@ -102488,7 +102608,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v8, v39
+; SI-NEXT: v_cvt_f32_f16_e32 v8, v38
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f32_f16_e32 v16, v55
@@ -102510,23 +102630,23 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v10, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v10, v38
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: v_cvt_f32_f16_e32 v10, v37
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v11, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v11, v37
+; SI-NEXT: v_cvt_f32_f16_e32 v11, v63
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -102655,7 +102775,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_cvt_f32_f16_e32 v25, v25
; SI-NEXT: s_waitcnt vmcnt(2)
@@ -102671,7 +102791,7 @@ define <16 x i64> @bitcast_v64f16_to_v16i64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -121431,341 +121551,366 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: v_readfirstlane_b32 s58, v10
-; SI-NEXT: v_readfirstlane_b32 s59, v9
-; SI-NEXT: v_readfirstlane_b32 s60, v8
-; SI-NEXT: v_readfirstlane_b32 s61, v7
-; SI-NEXT: v_readfirstlane_b32 s76, v0
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:324
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:316
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:300
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:292
-; SI-NEXT: v_readfirstlane_b32 s46, v30
-; SI-NEXT: v_readfirstlane_b32 s6, v29
-; SI-NEXT: v_readfirstlane_b32 s7, v28
-; SI-NEXT: v_readfirstlane_b32 s8, v27
-; SI-NEXT: v_readfirstlane_b32 s9, v26
-; SI-NEXT: v_readfirstlane_b32 s10, v25
-; SI-NEXT: v_readfirstlane_b32 s11, v24
-; SI-NEXT: v_readfirstlane_b32 s12, v23
-; SI-NEXT: v_readfirstlane_b32 s13, v22
-; SI-NEXT: v_readfirstlane_b32 s14, v21
-; SI-NEXT: v_readfirstlane_b32 s15, v20
-; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:16
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:24
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:32
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:144
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:152
+; SI-NEXT: v_readfirstlane_b32 s75, v0
; SI-NEXT: v_readfirstlane_b32 s41, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:332
+; SI-NEXT: v_readfirstlane_b32 s15, v20
; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:16
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:24
-; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:32
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:40
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:96
+; SI-NEXT: v_readfirstlane_b32 s40, v19
+; SI-NEXT: v_readfirstlane_b32 s7, v28
+; SI-NEXT: v_readfirstlane_b32 s9, v26
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:192
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:200
; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:208
; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:216
+; SI-NEXT: v_readfirstlane_b32 s76, v30
+; SI-NEXT: v_readfirstlane_b32 s6, v29
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:160
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:168
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:176
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s46, v13
+; SI-NEXT: v_readfirstlane_b32 s47, v12
+; SI-NEXT: v_readfirstlane_b32 s56, v11
+; SI-NEXT: v_readfirstlane_b32 s45, v14
+; SI-NEXT: v_readfirstlane_b32 s43, v16
+; SI-NEXT: v_readfirstlane_b32 s44, v15
+; SI-NEXT: v_readfirstlane_b32 s74, v1
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:60
; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:224
; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:232
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:136
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:240
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:144
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:152
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:176
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:184
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:192
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:200
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:208
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:216
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:224
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:232
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:240
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:248
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:256
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:264
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:272
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:280
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:288
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:296
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:304
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:312
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:320
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:328
-; SI-NEXT: v_readfirstlane_b32 s43, v16
-; SI-NEXT: v_readfirstlane_b32 s44, v15
-; SI-NEXT: v_readfirstlane_b32 s45, v14
-; SI-NEXT: v_readfirstlane_b32 s47, v13
-; SI-NEXT: v_readfirstlane_b32 s56, v12
-; SI-NEXT: v_readfirstlane_b32 s57, v11
-; SI-NEXT: v_readfirstlane_b32 s62, v6
-; SI-NEXT: v_readfirstlane_b32 s63, v5
-; SI-NEXT: v_readfirstlane_b32 s72, v4
-; SI-NEXT: v_readfirstlane_b32 s73, v3
-; SI-NEXT: v_readfirstlane_b32 s74, v2
-; SI-NEXT: v_readfirstlane_b32 s75, v1
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:248
+; SI-NEXT: v_readfirstlane_b32 s8, v27
+; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: v_readfirstlane_b32 s11, v24
+; SI-NEXT: v_readfirstlane_b32 s12, v23
+; SI-NEXT: v_readfirstlane_b32 s13, v22
+; SI-NEXT: v_readfirstlane_b32 s14, v21
+; SI-NEXT: v_readfirstlane_b32 s57, v10
+; SI-NEXT: v_readfirstlane_b32 s58, v9
+; SI-NEXT: v_readfirstlane_b32 s59, v8
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_readfirstlane_b32 s62, v5
+; SI-NEXT: v_readfirstlane_b32 s72, v3
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:172
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:180
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:212
+; SI-NEXT: v_readfirstlane_b32 s60, v7
+; SI-NEXT: v_readfirstlane_b32 s63, v4
+; SI-NEXT: v_readfirstlane_b32 s73, v2
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:148
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v16, 8, v33
-; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v34
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v42
-; SI-NEXT: v_lshlrev_b32_e32 v14, 24, v43
-; SI-NEXT: v_lshlrev_b32_e32 v51, 24, v45
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v47
-; SI-NEXT: v_lshlrev_b32_e32 v33, 24, v59
-; SI-NEXT: v_lshlrev_b32_e32 v42, 24, v61
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v20
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v21
-; SI-NEXT: v_lshlrev_b32_e32 v22, 24, v22
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v17
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
+; SI-NEXT: s_and_b64 s[4:5], vcc, exec
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
+; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v37
+; SI-NEXT: v_lshlrev_b32_e32 v50, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v49
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v39
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v38
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v36
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v46
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v33
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v56
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v58
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:84
+; SI-NEXT: v_lshlrev_b32_e32 v45, 8, v34
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v48
+; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 24, v28
+; SI-NEXT: v_lshlrev_b32_e32 v24, 8, v57
+; SI-NEXT: v_lshlrev_b32_e32 v9, 24, v58
+; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v30
+; SI-NEXT: v_lshlrev_b32_e32 v33, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v30, 24, v56
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v59
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v60
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v61
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v62
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v60
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v53
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v62
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:92
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:100
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:108
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v55
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v54
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v53
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(14) expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v52
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:132
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v40
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v30
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v43
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v27
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v44
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v50
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v46
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:156
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v49
-; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v18
-; SI-NEXT: v_lshlrev_b32_e32 v17, 8, v36
-; SI-NEXT: v_lshlrev_b32_e32 v18, 24, v41
-; SI-NEXT: v_lshlrev_b32_e32 v44, 24, v57
-; SI-NEXT: v_lshlrev_b32_e32 v55, 24, v24
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v25
-; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v29
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:264
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:272
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:280
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:252
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:260
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:276
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v46, 8, v39
-; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v37
-; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v35
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v32
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:284
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:276
-; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:268
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:260
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:252
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:244
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:228
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:220
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:196
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:180
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:172
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:164
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:156
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:220
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:228
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:236
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:148
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:140
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:132
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:20
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:12
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:4
-; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_lshlrev_b32_e32 v31, 8, v31
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:244
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v11
+; SI-NEXT: s_waitcnt vmcnt(11)
+; SI-NEXT: v_lshlrev_b32_e32 v32, 24, v12
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v13
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:288
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:296
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:304
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:312
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:284
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:292
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:308
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v29, 24, v29
+; SI-NEXT: s_waitcnt vmcnt(7)
+; SI-NEXT: v_lshlrev_b32_e32 v12, 8, v12
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v36
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshlrev_b32_e32 v42, 8, v38
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v54
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:320
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:324
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v54
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v46
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v59, off, s[0:3], s32 offset:12
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; SI-NEXT: s_cbranch_scc0 .LBB75_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_mov_b32_e32 v40, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; SI-NEXT: s_and_b32 s4, s46, 0xff
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v6, v15, v6
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_or_b32_e32 v38, v38, v11
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v17
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v16
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_or_b32_e32 v12, v12, v38
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v33, v4
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_or_b32_e32 v2, v42, v2
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v35
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_or_b32_e32 v60, v37, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v59
+; SI-NEXT: v_or_b32_e32 v35, v35, v50
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v50, 0xffff, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xff, v58
+; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v35
+; SI-NEXT: v_or_b32_e32 v48, v48, v58
+; SI-NEXT: v_mov_b32_e32 v58, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_and_b32 s4, s76, 0xff
+; SI-NEXT: v_or_b32_e32 v51, s4, v51
+; SI-NEXT: v_and_b32_e32 v51, 0xffff, v51
+; SI-NEXT: v_mov_b32_e32 v62, v32
+; SI-NEXT: v_mov_b32_e32 v32, v13
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v57
+; SI-NEXT: v_mov_b32_e32 v35, v11
+; SI-NEXT: v_or_b32_e32 v11, v51, v60
+; SI-NEXT: v_mov_b32_e32 v59, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v56
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_or_b32_e32 v12, v50, v48
+; SI-NEXT: v_mov_b32_e32 v37, v42
+; SI-NEXT: v_mov_b32_e32 v61, v40
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_and_b32_e32 v18, 0xff, v18
+; SI-NEXT: v_and_b32_e32 v19, 0xff, v19
+; SI-NEXT: v_and_b32_e32 v20, 0xff, v20
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: v_and_b32_e32 v23, 0xff, v23
+; SI-NEXT: v_or_b32_e32 v1, v1, v4
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v20, v0
-; SI-NEXT: s_lshl_b32 s5, s17, 8
-; SI-NEXT: s_lshl_b32 s77, s19, 24
-; SI-NEXT: s_lshl_b32 s78, s23, 24
-; SI-NEXT: s_lshl_b32 s79, s27, 24
-; SI-NEXT: s_lshl_b32 s88, s75, 24
-; SI-NEXT: s_lshl_b32 s89, s63, 24
-; SI-NEXT: s_lshl_b32 s90, s59, 24
-; SI-NEXT: s_lshl_b32 s91, s47, 24
-; SI-NEXT: s_lshl_b32 s92, s42, 24
-; SI-NEXT: s_lshl_b32 s93, s14, 24
-; SI-NEXT: s_lshl_b32 s94, s10, 24
-; SI-NEXT: s_lshl_b32 s95, s6, 24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v7, s4, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_or_b32_e32 v11, v7, v6
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v63
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v7, v18, v7
-; SI-NEXT: v_mov_b32_e32 v6, v13
-; SI-NEXT: v_or_b32_e32 v13, v5, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xff, v62
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v60
-; SI-NEXT: v_or_b32_e32 v5, v5, v19
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v14, v7
-; SI-NEXT: v_mov_b32_e32 v60, v14
-; SI-NEXT: v_or_b32_e32 v14, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v57
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v0, v2, v0
+; SI-NEXT: v_and_b32_e32 v27, 0xff, v27
+; SI-NEXT: v_and_b32_e32 v22, 0xff, v22
+; SI-NEXT: v_or_b32_e32 v23, v26, v23
+; SI-NEXT: v_or_b32_e32 v26, v1, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v55
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v61
+; SI-NEXT: v_or_b32_e32 v27, v27, v58
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
+; SI-NEXT: v_or_b32_e32 v22, v30, v22
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v62, v1
+; SI-NEXT: v_or_b32_e32 v22, v27, v22
+; SI-NEXT: v_or_b32_e32 v27, v0, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v59
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v1, v29, v1
+; SI-NEXT: v_or_b32_e32 v28, v0, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
+; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s5, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s77, s19, 24
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s5, s77, s5
; SI-NEXT: s_or_b32 s4, s4, s5
@@ -121774,6 +121919,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s5, s5, s77
; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s77, s78, s77
; SI-NEXT: s_or_b32 s5, s5, s77
@@ -121782,38 +121928,48 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s26, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
+; SI-NEXT: s_lshl_b32 s79, s27, 24
; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_or_b32 s78, s79, s78
; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_and_b32 s78, s28, 0xff
; SI-NEXT: s_lshl_b32 s79, s29, 8
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_mov_b32_e32 v51, v33
+; SI-NEXT: v_or_b32_e32 v13, v13, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s76, 0xff
+; SI-NEXT: s_and_b32 s79, s75, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s78, s78, 0xffff
; SI-NEXT: s_or_b32 s79, s88, s79
; SI-NEXT: s_or_b32 s78, s78, s79
-; SI-NEXT: s_and_b32 s79, s74, 0xff
-; SI-NEXT: s_lshl_b32 s88, s73, 8
+; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_lshl_b32 s88, s72, 8
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s72, 0xff
+; SI-NEXT: s_and_b32 s88, s63, 0xff
; SI-NEXT: s_lshl_b32 s88, s88, 16
+; SI-NEXT: s_lshl_b32 s89, s62, 24
; SI-NEXT: s_and_b32 s79, s79, 0xffff
; SI-NEXT: s_or_b32 s88, s89, s88
; SI-NEXT: s_or_b32 s79, s79, s88
-; SI-NEXT: s_and_b32 s88, s62, 0xff
-; SI-NEXT: s_lshl_b32 s89, s61, 8
+; SI-NEXT: s_and_b32 s88, s61, 0xff
+; SI-NEXT: s_lshl_b32 s89, s60, 8
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s60, 0xff
+; SI-NEXT: s_and_b32 s89, s59, 0xff
; SI-NEXT: s_lshl_b32 s89, s89, 16
+; SI-NEXT: s_lshl_b32 s90, s58, 24
; SI-NEXT: s_and_b32 s88, s88, 0xffff
; SI-NEXT: s_or_b32 s89, s90, s89
; SI-NEXT: s_or_b32 s88, s88, s89
-; SI-NEXT: s_and_b32 s89, s58, 0xff
-; SI-NEXT: s_lshl_b32 s90, s57, 8
+; SI-NEXT: s_and_b32 s89, s57, 0xff
+; SI-NEXT: s_lshl_b32 s90, s56, 8
; SI-NEXT: s_or_b32 s89, s89, s90
-; SI-NEXT: s_and_b32 s90, s56, 0xff
+; SI-NEXT: s_and_b32 s90, s47, 0xff
; SI-NEXT: s_lshl_b32 s90, s90, 16
+; SI-NEXT: s_lshl_b32 s91, s46, 24
; SI-NEXT: s_and_b32 s89, s89, 0xffff
; SI-NEXT: s_or_b32 s90, s91, s90
; SI-NEXT: s_or_b32 s89, s89, s90
@@ -121822,6 +121978,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s90, s90, s91
; SI-NEXT: s_and_b32 s91, s43, 0xff
; SI-NEXT: s_lshl_b32 s91, s91, 16
+; SI-NEXT: s_lshl_b32 s92, s42, 24
; SI-NEXT: s_and_b32 s90, s90, 0xffff
; SI-NEXT: s_or_b32 s91, s92, s91
; SI-NEXT: s_or_b32 s90, s90, s91
@@ -121830,6 +121987,8 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s91, s91, s92
; SI-NEXT: s_and_b32 s92, s15, 0xff
; SI-NEXT: s_lshl_b32 s92, s92, 16
+; SI-NEXT: s_lshl_b32 s93, s14, 24
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
; SI-NEXT: s_and_b32 s91, s91, 0xffff
; SI-NEXT: s_or_b32 s92, s93, s92
; SI-NEXT: s_or_b32 s91, s91, s92
@@ -121838,180 +121997,178 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s11, 0xff
; SI-NEXT: s_lshl_b32 s93, s93, 16
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v51, v7
-; SI-NEXT: v_or_b32_e32 v15, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v32
+; SI-NEXT: s_lshl_b32 s94, s10, 24
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: s_and_b32 s92, s92, 0xffff
; SI-NEXT: s_or_b32 s93, s94, s93
+; SI-NEXT: v_or_b32_e32 v8, v8, v24
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: s_or_b32 s92, s92, s93
; SI-NEXT: s_and_b32 s93, s9, 0xff
; SI-NEXT: s_lshl_b32 s94, s8, 8
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
+; SI-NEXT: v_or_b32_e32 v5, v9, v5
; SI-NEXT: s_or_b32 s93, s93, s94
; SI-NEXT: s_and_b32 s94, s7, 0xff
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v21, 0xff, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xff, v25
+; SI-NEXT: v_or_b32_e32 v24, v8, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v6
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_mov_b32_e32 v50, v33
+; SI-NEXT: v_or_b32_e32 v14, v33, v14
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v13, v13, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v47
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: s_lshl_b32 s94, s94, 16
+; SI-NEXT: s_lshl_b32 s95, s6, 24
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_or_b32_e32 v25, v25, v39
+; SI-NEXT: v_or_b32_e32 v5, v5, v10
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: s_and_b32 s93, s93, 0xffff
; SI-NEXT: s_or_b32 s94, s95, s94
-; SI-NEXT: s_or_b32 s93, s93, s94
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v45
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v48, v7
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_or_b32_e32 v16, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v43
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v52
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v7, v44, v7
-; SI-NEXT: v_or_b32_e32 v17, v5, v7
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v41
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v21, v63, v21
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v25
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_or_b32_e32 v18, v5, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v5, s88
-; SI-NEXT: v_mov_b32_e32 v7, s90
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v19, v3, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v3, v7, v3
+; SI-NEXT: s_or_b32 s93, s93, s94
+; SI-NEXT: v_or_b32_e32 v23, v25, v23
+; SI-NEXT: v_or_b32_e32 v25, v5, v3
+; SI-NEXT: v_mov_b32_e32 v2, s77
; SI-NEXT: v_mov_b32_e32 v3, s78
; SI-NEXT: v_mov_b32_e32 v4, s79
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v20, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v54
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v5, s88
+; SI-NEXT: v_mov_b32_e32 v6, s89
+; SI-NEXT: v_mov_b32_e32 v7, s90
+; SI-NEXT: v_mov_b32_e32 v8, s91
+; SI-NEXT: v_mov_b32_e32 v9, s92
+; SI-NEXT: v_mov_b32_e32 v10, s93
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v53
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v21, v1
-; SI-NEXT: v_or_b32_e32 v21, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v50
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v49
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v22, v1
-; SI-NEXT: v_or_b32_e32 v22, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v39
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v23
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v23, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v27
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v48, v33
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v46
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
+; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_or_b32_e32 v33, v42, v33
+; SI-NEXT: v_mov_b32_e32 v46, v44
+; SI-NEXT: v_mov_b32_e32 v44, v42
+; SI-NEXT: v_mov_b32_e32 v42, v38
+; SI-NEXT: v_or_b32_e32 v14, v14, v33
+; SI-NEXT: v_mov_b32_e32 v38, v40
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v40
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v33, v33, v45
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v24
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v55, v1
-; SI-NEXT: v_or_b32_e32 v24, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v59, v1
-; SI-NEXT: v_or_b32_e32 v25, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v29
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v26
-; SI-NEXT: v_or_b32_e32 v0, v0, v31
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v56, v1
-; SI-NEXT: v_or_b32_e32 v26, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v30
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_and_b32_e32 v1, 0xff, v35
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v27, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v61
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_or_b32_e32 v28, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v6
-; SI-NEXT: v_mov_b32_e32 v6, s89
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_or_b32_e32 v1, v43, v1
; SI-NEXT: v_or_b32_e32 v29, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v10
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v0, v0, v46
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v32
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v34
+; SI-NEXT: v_or_b32_e32 v0, v0, v37
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v36, v1
; SI-NEXT: v_or_b32_e32 v30, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v8
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v40
-; SI-NEXT: v_or_b32_e32 v0, v0, v34
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v42
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v52
+; SI-NEXT: v_or_b32_e32 v0, v0, v46
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v37, v1
+; SI-NEXT: v_or_b32_e32 v1, v54, v1
; SI-NEXT: v_or_b32_e32 v31, v0, v1
; SI-NEXT: v_mov_b32_e32 v0, s4
; SI-NEXT: v_mov_b32_e32 v1, s5
-; SI-NEXT: v_mov_b32_e32 v2, s77
-; SI-NEXT: v_mov_b32_e32 v8, s91
-; SI-NEXT: v_mov_b32_e32 v9, s92
-; SI-NEXT: v_mov_b32_e32 v10, s93
; SI-NEXT: s_mov_b64 s[4:5], 0
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v15, v15, v40
+; SI-NEXT: v_or_b32_e32 v15, v33, v15
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v16, v16, v40
+; SI-NEXT: v_or_b32_e32 v16, v33, v16
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v33, 0xff, v33
+; SI-NEXT: v_or_b32_e32 v33, v33, v41
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v17, v17, v40
+; SI-NEXT: v_or_b32_e32 v17, v33, v17
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v18
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v18, v18, v40
+; SI-NEXT: v_or_b32_e32 v18, v33, v18
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v19, v19, v40
+; SI-NEXT: v_or_b32_e32 v19, v33, v19
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v20, v20, v40
+; SI-NEXT: v_or_b32_e32 v20, v33, v20
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v40, 0xff, v33
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_or_b32_e32 v33, v40, v33
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v33
+; SI-NEXT: v_or_b32_e32 v21, v33, v21
; SI-NEXT: s_branch .LBB75_3
; SI-NEXT: .LBB75_2:
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v38, v17
-; SI-NEXT: v_mov_b32_e32 v63, v18
-; SI-NEXT: v_mov_b32_e32 v62, v19
-; SI-NEXT: v_mov_b32_e32 v60, v14
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15_vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23_vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31
; SI-NEXT: .LBB75_3: ; %Flow
@@ -122020,9 +122177,9 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB75_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
-; SI-NEXT: s_add_i32 s46, s46, 3
-; SI-NEXT: s_and_b32 s4, s46, 0xff
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; SI-NEXT: s_add_i32 s76, s76, 3
+; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_lshl_b32 s5, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
@@ -122033,14 +122190,14 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s76, s76, 3
-; SI-NEXT: s_add_i32 s74, s74, 3
-; SI-NEXT: s_add_i32 s72, s72, 3
-; SI-NEXT: s_add_i32 s62, s62, 3
-; SI-NEXT: s_add_i32 s60, s60, 3
-; SI-NEXT: s_add_i32 s58, s58, 3
-; SI-NEXT: s_add_i32 s56, s56, 3
-; SI-NEXT: s_and_b32 s22, s56, 0xff
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s63, s63, 3
+; SI-NEXT: s_add_i32 s61, s61, 3
+; SI-NEXT: s_add_i32 s59, s59, 3
+; SI-NEXT: s_add_i32 s57, s57, 3
+; SI-NEXT: s_add_i32 s47, s47, 3
+; SI-NEXT: s_and_b32 s22, s47, 0xff
; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_add_i32 s45, s45, 3
; SI-NEXT: s_add_i32 s43, s43, 3
@@ -122078,22 +122235,22 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
; SI-NEXT: v_mov_b32_e32 v9, s10
; SI-NEXT: v_mov_b32_e32 v10, s6
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, s4, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v1, v1, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -122125,38 +122282,38 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: s_and_b32 s17, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s29, 8
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s19, s76, 0xff
+; SI-NEXT: s_and_b32 s19, s75, 0xff
; SI-NEXT: s_addk_i32 s17, 0x300
-; SI-NEXT: s_lshl_b32 s18, s75, 24
+; SI-NEXT: s_lshl_b32 s18, s74, 24
; SI-NEXT: s_lshl_b32 s19, s19, 16
; SI-NEXT: s_and_b32 s17, s17, 0xffff
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_or_b32 s17, s18, s17
-; SI-NEXT: s_and_b32 s18, s74, 0xff
-; SI-NEXT: s_lshl_b32 s19, s73, 8
+; SI-NEXT: s_and_b32 s18, s73, 0xff
+; SI-NEXT: s_lshl_b32 s19, s72, 8
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_and_b32 s20, s63, 0xff
; SI-NEXT: s_addk_i32 s18, 0x300
-; SI-NEXT: s_lshl_b32 s19, s63, 24
+; SI-NEXT: s_lshl_b32 s19, s62, 24
; SI-NEXT: s_lshl_b32 s20, s20, 16
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_or_b32 s18, s19, s18
-; SI-NEXT: s_and_b32 s19, s62, 0xff
-; SI-NEXT: s_lshl_b32 s20, s61, 8
+; SI-NEXT: s_and_b32 s19, s61, 0xff
+; SI-NEXT: s_lshl_b32 s20, s60, 8
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s21, s60, 0xff
+; SI-NEXT: s_and_b32 s21, s59, 0xff
; SI-NEXT: s_addk_i32 s19, 0x300
-; SI-NEXT: s_lshl_b32 s20, s59, 24
+; SI-NEXT: s_lshl_b32 s20, s58, 24
; SI-NEXT: s_lshl_b32 s21, s21, 16
; SI-NEXT: s_and_b32 s19, s19, 0xffff
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s20, s58, 0xff
-; SI-NEXT: s_lshl_b32 s21, s57, 8
+; SI-NEXT: s_and_b32 s20, s57, 0xff
+; SI-NEXT: s_lshl_b32 s21, s56, 8
; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_addk_i32 s20, 0x300
-; SI-NEXT: s_lshl_b32 s21, s47, 24
+; SI-NEXT: s_lshl_b32 s21, s46, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_or_b32 s20, s21, s20
@@ -122176,7 +122333,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_or_b32_e32 v2, v2, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x300, v2
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: s_or_b32 s22, s23, s22
@@ -122204,333 +122361,330 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: v_or_b32_e32 v0, v3, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; SI-NEXT: v_mov_b32_e32 v3, s17
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v38, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v51, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v63, v0
+; SI-NEXT: v_or_b32_e32 v0, v50, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v1, v62, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v1, v48, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v0, v60, v0
+; SI-NEXT: v_or_b32_e32 v0, v44, v0
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
+; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v38
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v16, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v17, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v18, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v20, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v21, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v22, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v1, v0, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x300, v1
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v24, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v25, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v26, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v27, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v28, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v29, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -122540,21 +122694,21 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v30, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v0, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -122579,7 +122733,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0)
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
;
; VI-LABEL: bitcast_v128i8_to_v16f64_scalar:
@@ -122604,8 +122758,9 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; VI-NEXT: v_readfirstlane_b32 s73, v3
+; VI-NEXT: v_readfirstlane_b32 s74, v2
; VI-NEXT: v_readfirstlane_b32 s47, v13
+; VI-NEXT: v_readfirstlane_b32 s72, v4
; VI-NEXT: v_readfirstlane_b32 s61, v7
; VI-NEXT: v_readfirstlane_b32 s62, v6
; VI-NEXT: v_readfirstlane_b32 s63, v5
@@ -122631,101 +122786,100 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
-; VI-NEXT: v_readfirstlane_b32 s72, v4
-; VI-NEXT: v_readfirstlane_b32 s74, v2
+; VI-NEXT: v_readfirstlane_b32 s73, v3
; VI-NEXT: v_readfirstlane_b32 s75, v1
-; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
+; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:316
+; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
+; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:308
; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:296
; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; VI-NEXT: s_waitcnt vmcnt(13)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: s_waitcnt vmcnt(10)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:320
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; VI-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:268
+; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:264
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:248
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
-; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
+; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:220
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; VI-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; VI-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; VI-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; VI-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:180
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:148
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:136
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; VI-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -122746,186 +122900,192 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:48
; VI-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; VI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
; VI-NEXT: s_cbranch_scc0 .LBB75_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: v_mov_b32_e32 v20, 0xc0c0004
-; VI-NEXT: v_mov_b32_e32 v4, s23
-; VI-NEXT: v_mov_b32_e32 v2, s19
-; VI-NEXT: v_mov_b32_e32 v3, s21
-; VI-NEXT: v_perm_b32 v4, s22, v4, v20
+; VI-NEXT: v_mov_b32_e32 v40, 0xc0c0004
+; VI-NEXT: v_mov_b32_e32 v3, s19
; VI-NEXT: v_mov_b32_e32 v13, s17
-; VI-NEXT: v_perm_b32 v2, s18, v2, v20
-; VI-NEXT: v_perm_b32 v3, s20, v3, v20
-; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_perm_b32 v13, s16, v13, v20
+; VI-NEXT: v_perm_b32 v3, s18, v3, v40
+; VI-NEXT: v_mov_b32_e32 v2, s23
+; VI-NEXT: v_perm_b32 v13, s16, v13, v40
+; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; VI-NEXT: v_mov_b32_e32 v42, s21
+; VI-NEXT: v_perm_b32 v2, s22, v2, v40
+; VI-NEXT: v_perm_b32 v42, s20, v42, v40
; VI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; VI-NEXT: v_mov_b32_e32 v19, v1
-; VI-NEXT: v_or_b32_e32 v1, v3, v4
+; VI-NEXT: v_mov_b32_e32 v20, v0
+; VI-NEXT: v_or_b32_e32 v0, v13, v3
; VI-NEXT: v_mov_b32_e32 v3, s27
-; VI-NEXT: v_mov_b32_e32 v22, v0
-; VI-NEXT: v_or_b32_e32 v0, v13, v2
+; VI-NEXT: v_mov_b32_e32 v19, v1
+; VI-NEXT: v_or_b32_e32 v1, v42, v2
; VI-NEXT: v_mov_b32_e32 v2, s25
-; VI-NEXT: v_perm_b32 v3, s26, v3, v20
-; VI-NEXT: v_perm_b32 v2, s24, v2, v20
+; VI-NEXT: v_perm_b32 v3, s26, v3, v40
+; VI-NEXT: v_mov_b32_e32 v59, v4
+; VI-NEXT: v_perm_b32 v2, s24, v2, v40
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; VI-NEXT: v_mov_b32_e32 v4, s75
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s76, v4, v20
-; VI-NEXT: v_mov_b32_e32 v44, v5
-; VI-NEXT: v_perm_b32 v3, s28, v3, v20
+; VI-NEXT: v_perm_b32 v4, s76, v4, v40
+; VI-NEXT: v_mov_b32_e32 v47, v5
+; VI-NEXT: v_perm_b32 v3, s28, v3, v40
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; VI-NEXT: v_mov_b32_e32 v5, s63
; VI-NEXT: v_or_b32_e32 v3, v3, v4
; VI-NEXT: v_mov_b32_e32 v4, s73
-; VI-NEXT: v_perm_b32 v5, s72, v5, v20
-; VI-NEXT: v_mov_b32_e32 v46, v6
-; VI-NEXT: v_perm_b32 v4, s74, v4, v20
+; VI-NEXT: v_perm_b32 v5, s72, v5, v40
+; VI-NEXT: v_mov_b32_e32 v57, v6
+; VI-NEXT: v_perm_b32 v4, s74, v4, v40
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; VI-NEXT: v_mov_b32_e32 v6, s59
; VI-NEXT: v_or_b32_e32 v4, v4, v5
; VI-NEXT: v_mov_b32_e32 v5, s61
-; VI-NEXT: v_perm_b32 v6, s60, v6, v20
-; VI-NEXT: v_mov_b32_e32 v45, v7
-; VI-NEXT: v_perm_b32 v5, s62, v5, v20
+; VI-NEXT: v_perm_b32 v6, s60, v6, v40
+; VI-NEXT: v_mov_b32_e32 v56, v7
+; VI-NEXT: v_perm_b32 v5, s62, v5, v40
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; VI-NEXT: v_mov_b32_e32 v7, s47
; VI-NEXT: v_or_b32_e32 v5, v5, v6
; VI-NEXT: v_mov_b32_e32 v6, s57
-; VI-NEXT: v_perm_b32 v7, s56, v7, v20
-; VI-NEXT: v_mov_b32_e32 v47, v40
-; VI-NEXT: v_mov_b32_e32 v40, v8
-; VI-NEXT: v_perm_b32 v6, s58, v6, v20
+; VI-NEXT: v_perm_b32 v7, s56, v7, v40
+; VI-NEXT: v_mov_b32_e32 v43, v8
+; VI-NEXT: v_perm_b32 v6, s58, v6, v40
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_mov_b32_e32 v7, s45
-; VI-NEXT: v_perm_b32 v8, s44, v8, v20
-; VI-NEXT: v_mov_b32_e32 v43, v9
-; VI-NEXT: v_perm_b32 v7, s46, v7, v20
+; VI-NEXT: v_perm_b32 v8, s44, v8, v40
+; VI-NEXT: v_mov_b32_e32 v46, v9
+; VI-NEXT: v_perm_b32 v7, s46, v7, v40
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s15
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s41
-; VI-NEXT: v_perm_b32 v9, s40, v9, v20
-; VI-NEXT: v_mov_b32_e32 v56, v41
-; VI-NEXT: v_mov_b32_e32 v41, v10
-; VI-NEXT: v_perm_b32 v8, s42, v8, v20
+; VI-NEXT: v_perm_b32 v9, s40, v9, v40
+; VI-NEXT: v_mov_b32_e32 v44, v10
+; VI-NEXT: v_perm_b32 v8, s42, v8, v40
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_or_b32_e32 v8, v8, v9
; VI-NEXT: v_mov_b32_e32 v9, s13
-; VI-NEXT: v_perm_b32 v10, s12, v10, v20
-; VI-NEXT: v_mov_b32_e32 v42, v11
-; VI-NEXT: v_perm_b32 v9, s14, v9, v20
+; VI-NEXT: v_perm_b32 v10, s12, v10, v40
+; VI-NEXT: v_mov_b32_e32 v58, v41
+; VI-NEXT: v_mov_b32_e32 v41, v11
+; VI-NEXT: v_perm_b32 v9, s14, v9, v40
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_or_b32_e32 v9, v9, v10
; VI-NEXT: v_mov_b32_e32 v10, s9
-; VI-NEXT: v_perm_b32 v11, s8, v11, v20
-; VI-NEXT: v_perm_b32 v10, s10, v10, v20
+; VI-NEXT: v_perm_b32 v11, s8, v11, v40
+; VI-NEXT: v_perm_b32 v10, s10, v10, v40
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
+; VI-NEXT: v_perm_b32 v18, v41, v18, s4
+; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; VI-NEXT: v_perm_b32 v19, v63, v19, s4
+; VI-NEXT: v_or_b32_e32 v18, v19, v18
+; VI-NEXT: v_perm_b32 v19, v22, v45, s4
+; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; VI-NEXT: v_perm_b32 v20, v62, v20, s4
+; VI-NEXT: v_or_b32_e32 v19, v20, v19
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: v_perm_b32 v12, v12, v61, s4
; VI-NEXT: v_perm_b32 v14, v52, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_perm_b32 v15, v48, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_perm_b32 v16, v50, v16, s4
-; VI-NEXT: v_perm_b32 v18, v42, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_perm_b32 v17, v35, v17, s4
-; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
-; VI-NEXT: v_perm_b32 v19, v63, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
-; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_perm_b32 v19, v23, v24, s4
-; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; VI-NEXT: v_perm_b32 v23, v34, v33, s4
-; VI-NEXT: v_perm_b32 v24, v41, v43, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v11, s6, v11, v20
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v22, v30, v31, s4
+; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: v_perm_b32 v11, s6, v11, v40
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: s_waitcnt vmcnt(2)
+; VI-NEXT: v_perm_b32 v20, v25, v20, s4
+; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v25, v56, v57, s4
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v20, v13, s4
+; VI-NEXT: v_perm_b32 v13, v40, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_e32 v11, v11, v13
; VI-NEXT: v_perm_b32 v13, v21, v60, s4
@@ -122933,88 +123093,84 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v27, v29, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; VI-NEXT: v_perm_b32 v20, v57, v28, s4
-; VI-NEXT: v_or_b32_e32 v13, v20, v13
-; VI-NEXT: v_perm_b32 v20, v55, v51, s4
-; VI-NEXT: v_or_b32_e32 v14, v20, v14
-; VI-NEXT: v_perm_b32 v20, v54, v53, s4
-; VI-NEXT: v_or_b32_e32 v15, v20, v15
-; VI-NEXT: v_perm_b32 v20, v39, v36, s4
-; VI-NEXT: v_or_b32_e32 v16, v20, v16
-; VI-NEXT: v_perm_b32 v20, v49, v38, s4
-; VI-NEXT: v_or_b32_e32 v17, v20, v17
-; VI-NEXT: v_perm_b32 v20, v62, v22, s4
-; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
-; VI-NEXT: v_perm_b32 v21, v25, v58, s4
-; VI-NEXT: v_perm_b32 v22, v30, v31, s4
-; VI-NEXT: v_perm_b32 v25, v45, v46, s4
-; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v26, v20, s4
-; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_perm_b32 v21, v26, v28, s4
+; VI-NEXT: v_or_b32_e32 v13, v21, v13
+; VI-NEXT: v_perm_b32 v21, v55, v32, s4
+; VI-NEXT: v_or_b32_e32 v14, v21, v14
+; VI-NEXT: v_perm_b32 v21, v54, v53, s4
+; VI-NEXT: v_or_b32_e32 v15, v21, v15
+; VI-NEXT: v_perm_b32 v21, v39, v36, s4
+; VI-NEXT: v_or_b32_e32 v16, v21, v16
+; VI-NEXT: v_perm_b32 v21, v49, v38, s4
+; VI-NEXT: v_or_b32_e32 v17, v21, v17
+; VI-NEXT: v_perm_b32 v21, v23, v24, s4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v23, v34, v33, s4
+; VI-NEXT: v_perm_b32 v24, v44, v46, s4
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v37, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v32, v22, s4
+; VI-NEXT: v_perm_b32 v22, v51, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v40, v23, s4
+; VI-NEXT: v_perm_b32 v23, v43, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v44, v24, s4
+; VI-NEXT: v_perm_b32 v24, v47, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v25, v26, s4
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v25
+; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v25
; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v26, v25, s4
-; VI-NEXT: v_or_b32_e32 v25, v25, v27
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v25, v59, v25, s4
+; VI-NEXT: v_or_b32_e32 v25, v25, v26
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v26, v26, v27, s4
+; VI-NEXT: v_perm_b32 v26, v26, v58, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_or_b32_e32 v26, v26, v28
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v27, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; VI-NEXT: v_or_b32_e32 v27, v27, v29
-; VI-NEXT: v_perm_b32 v29, v47, v56, s4
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v28, v59, v28, s4
+; VI-NEXT: v_perm_b32 v28, v29, v28, s4
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; VI-NEXT: s_waitcnt vmcnt(0)
+; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v28, v29, v28
; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
-; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v30, v31, v30, s4
@@ -123029,16 +123185,16 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v31, v32, v31, s4
; VI-NEXT: v_or_b32_e32 v30, v31, v30
-; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v31, v32, v31, s4
-; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; VI-NEXT: v_perm_b32 v31, v31, v32, s4
+; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v31
+; VI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v32, v33, v32, s4
-; VI-NEXT: v_or_b32_e32 v31, v32, v31
+; VI-NEXT: v_perm_b32 v31, v33, v31, s4
+; VI-NEXT: v_or_b32_e32 v31, v31, v32
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB75_3
; VI-NEXT: .LBB75_2:
@@ -123150,7 +123306,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x300, v10
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; VI-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_add_u32_e32 v0, vcc, 0x3000000, v0
@@ -123165,12 +123321,12 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_add_u32_e32 v9, vcc, 0x3000000, v9
; VI-NEXT: v_add_u32_e32 v10, vcc, 0x3000000, v10
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v12, s6, v11, v12
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v12
+; VI-NEXT: v_perm_b32 v11, s6, v11, v12
+; VI-NEXT: v_add_u32_e32 v12, vcc, 0x300, v11
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v11
-; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v11, v13, v11, s4
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
@@ -123181,57 +123337,57 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v12
; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v13, v13, v12, s4
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
+; VI-NEXT: v_perm_b32 v12, v13, v12, s4
+; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v12
-; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v12, v14, v12, s4
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v12, v12, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v12, vcc, 0x3000000, v12
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v14, v14, v13, s4
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
+; VI-NEXT: v_perm_b32 v13, v14, v13, s4
+; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v13
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v13
-; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v15, v13, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x3000000, v13
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v15, v15, v14, s4
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v15
+; VI-NEXT: v_perm_b32 v14, v15, v14, s4
+; VI-NEXT: v_add_u32_e32 v15, vcc, 0x300, v14
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v14, v16, v14, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v14, v14, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x3000000, v14
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v16, vcc, 3, v15
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v16, v16, v15, s4
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v16
+; VI-NEXT: v_perm_b32 v15, v16, v15, s4
+; VI-NEXT: v_add_u32_e32 v16, vcc, 0x300, v15
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v15
; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
@@ -123239,151 +123395,151 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_perm_b32 v15, v17, v15, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; VI-NEXT: v_or_b32_sdwa v15, v15, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v15, vcc, 0x3000000, v15
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v17, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v17, v17, v16, s4
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v17
+; VI-NEXT: v_perm_b32 v16, v17, v16, s4
+; VI-NEXT: v_add_u32_e32 v17, vcc, 0x300, v16
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v16, v18, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; VI-NEXT: v_or_b32_sdwa v16, v16, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v16, vcc, 0x3000000, v16
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v18, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v18, v18, v17, s4
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v18
+; VI-NEXT: v_perm_b32 v17, v18, v17, s4
+; VI-NEXT: v_add_u32_e32 v18, vcc, 0x300, v17
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v17
-; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v17, v19, v17, s4
; VI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; VI-NEXT: v_or_b32_sdwa v17, v17, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v17, vcc, 0x3000000, v17
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v19, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v19, v19, v18, s4
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v19
+; VI-NEXT: v_perm_b32 v18, v19, v18, s4
+; VI-NEXT: v_add_u32_e32 v19, vcc, 0x300, v18
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v18
-; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v18, v20, v18, s4
; VI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; VI-NEXT: v_or_b32_sdwa v18, v18, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v18, vcc, 0x3000000, v18
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v20, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v20, v20, v19, s4
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v20
+; VI-NEXT: v_perm_b32 v19, v20, v19, s4
+; VI-NEXT: v_add_u32_e32 v20, vcc, 0x300, v19
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v19
-; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v19, v21, v19, s4
; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; VI-NEXT: v_or_b32_sdwa v19, v19, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v19, vcc, 0x3000000, v19
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v21, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v21, v21, v20, s4
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v21
+; VI-NEXT: v_perm_b32 v20, v21, v20, s4
+; VI-NEXT: v_add_u32_e32 v21, vcc, 0x300, v20
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v20
-; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v20, v22, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; VI-NEXT: v_or_b32_sdwa v20, v20, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v20, vcc, 0x3000000, v20
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v22, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v22, v22, v21, s4
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v22
+; VI-NEXT: v_perm_b32 v21, v22, v21, s4
+; VI-NEXT: v_add_u32_e32 v22, vcc, 0x300, v21
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v21
-; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v21, v23, v21, s4
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_or_b32_sdwa v21, v21, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v21, vcc, 0x3000000, v21
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v23, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v23, v23, v22, s4
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v23
+; VI-NEXT: v_perm_b32 v22, v23, v22, s4
+; VI-NEXT: v_add_u32_e32 v23, vcc, 0x300, v22
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v22
-; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v22, v24, v22, s4
; VI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; VI-NEXT: v_or_b32_sdwa v22, v22, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v22, vcc, 0x3000000, v22
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v24, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v24, v24, v23, s4
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v24
+; VI-NEXT: v_perm_b32 v23, v24, v23, s4
+; VI-NEXT: v_add_u32_e32 v24, vcc, 0x300, v23
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v23
-; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v23, v25, v23, s4
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_or_b32_sdwa v23, v23, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v23, vcc, 0x3000000, v23
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v25, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_perm_b32 v25, v25, v24, s4
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v25
+; VI-NEXT: v_perm_b32 v24, v25, v24, s4
+; VI-NEXT: v_add_u32_e32 v25, vcc, 0x300, v24
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v24
-; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v24, v26, v24, s4
; VI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; VI-NEXT: v_or_b32_sdwa v24, v24, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v24, vcc, 0x3000000, v24
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v26, vcc, 3, v25
@@ -123391,55 +123547,55 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v26, v25, s4
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x300, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v25
-; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v25, v27, v25, s4
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_or_b32_sdwa v25, v25, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v25, vcc, 0x3000000, v25
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v27, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v27, v26, s4
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x300, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v26
-; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v26, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; VI-NEXT: v_or_b32_sdwa v26, v26, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v26, vcc, 0x3000000, v26
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v28, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v28, v27, s4
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x300, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v27
-; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v27, v29, v27, s4
; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; VI-NEXT: v_or_b32_sdwa v27, v27, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v27, vcc, 0x3000000, v27
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v29, vcc, 3, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v28, v29, v28, s4
; VI-NEXT: v_add_u32_e32 v29, vcc, 0x300, v28
-; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v28
; VI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
@@ -123447,11 +123603,11 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; VI-NEXT: v_perm_b32 v28, v30, v28, s4
; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; VI-NEXT: v_or_b32_sdwa v28, v28, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; VI-NEXT: v_add_u32_e32 v28, vcc, 0x3000000, v28
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_add_u32_e32 v30, vcc, 3, v29
-; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v29, v30, v29, s4
; VI-NEXT: v_add_u32_e32 v30, vcc, 0x300, v29
@@ -123538,8 +123694,9 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332
-; GFX9-NEXT: v_readfirstlane_b32 s73, v3
-; GFX9-NEXT: v_readfirstlane_b32 s47, v13
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s72, v4
+; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s61, v7
; GFX9-NEXT: v_readfirstlane_b32 s62, v6
; GFX9-NEXT: v_readfirstlane_b32 s63, v5
@@ -123556,31 +123713,28 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_readfirstlane_b32 s13, v23
; GFX9-NEXT: v_readfirstlane_b32 s14, v22
; GFX9-NEXT: v_readfirstlane_b32 s15, v21
-; GFX9-NEXT: v_readfirstlane_b32 s40, v20
; GFX9-NEXT: v_readfirstlane_b32 s41, v19
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
+; GFX9-NEXT: v_readfirstlane_b32 s47, v13
; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
-; GFX9-NEXT: v_readfirstlane_b32 s72, v4
-; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s73, v3
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
-; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
-; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:324
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:320
-; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:316
-; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:312
+; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:320
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:316
+; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:312
; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:308
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:304
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:300
-; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:296
+; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:296
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:292
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:288
; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:284
-; GFX9-NEXT: s_waitcnt vmcnt(13)
+; GFX9-NEXT: s_waitcnt vmcnt(11)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:4
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -123590,82 +123744,86 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v2, off, s[0:3], s32 offset:328
+; GFX9-NEXT: s_nop 0
+; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:324
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:280
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:276
-; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:272
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:272
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:268
-; GFX9-NEXT: buffer_load_ushort v3, off, s[0:3], s32 offset:264
+; GFX9-NEXT: buffer_load_ushort v4, off, s[0:3], s32 offset:264
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:260
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:256
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:252
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v13, off, s[0:3], s32 offset:248
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:248
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:244
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:240
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:236
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:232
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:232
+; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:228
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:224
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:220
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:216
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v5, off, s[0:3], s32 offset:212
; GFX9-NEXT: buffer_load_ushort v6, off, s[0:3], s32 offset:208
; GFX9-NEXT: buffer_load_ushort v7, off, s[0:3], s32 offset:204
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:200
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v8, off, s[0:3], s32 offset:196
; GFX9-NEXT: buffer_load_ushort v9, off, s[0:3], s32 offset:192
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:188
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:184
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v51, off, s[0:3], s32 offset:180
-; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:176
+; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:176
; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:172
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:168
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:164
; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:160
; GFX9-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:156
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:152
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:148
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:144
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:140
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:136
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:132
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:148
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:144
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:140
+; GFX9-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:136
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v0, off, s[0:3], s32 offset:128
; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:120
@@ -123673,108 +123831,109 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:108
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:104
-; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:100
; GFX9-NEXT: buffer_load_ushort v38, off, s[0:3], s32 offset:96
-; GFX9-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v50, off, s[0:3], s32 offset:80
; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v15, off, s[0:3], s32 offset:72
-; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:68
-; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:64
-; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:60
-; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:56
+; GFX9-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:68
+; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:64
+; GFX9-NEXT: buffer_load_ushort v32, off, s[0:3], s32 offset:60
+; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:56
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v55, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v14, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:28
; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:12
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(55)
-; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(54)
-; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(51)
-; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(49)
-; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(52)
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(50)
-; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(61)
+; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; GFX9-NEXT: s_waitcnt vmcnt(60)
-; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(61)
-; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
+; GFX9-NEXT: s_waitcnt vmcnt(62)
+; GFX9-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:720 ; 4-byte Folded Spill
; GFX9-NEXT: s_waitcnt vmcnt(62)
; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:724 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Spill
@@ -123794,7 +123953,8 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v19, v1
; GFX9-NEXT: v_or_b32_e32 v1, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v3, s27
-; GFX9-NEXT: v_mov_b32_e32 v22, v0
+; GFX9-NEXT: v_mov_b32_e32 v46, v42
+; GFX9-NEXT: v_mov_b32_e32 v42, v0
; GFX9-NEXT: v_or_b32_e32 v0, v13, v2
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v20
@@ -123804,28 +123964,28 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_e32 v2, v2, v3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v20
-; GFX9-NEXT: v_mov_b32_e32 v44, v5
+; GFX9-NEXT: v_mov_b32_e32 v45, v5
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s63
; GFX9-NEXT: v_or_b32_e32 v3, v3, v4
; GFX9-NEXT: v_mov_b32_e32 v4, s73
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v20
-; GFX9-NEXT: v_mov_b32_e32 v46, v6
+; GFX9-NEXT: v_mov_b32_e32 v56, v6
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s59
; GFX9-NEXT: v_or_b32_e32 v4, v4, v5
; GFX9-NEXT: v_mov_b32_e32 v5, s61
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v20
-; GFX9-NEXT: v_mov_b32_e32 v45, v7
+; GFX9-NEXT: v_mov_b32_e32 v47, v7
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s47
; GFX9-NEXT: v_or_b32_e32 v5, v5, v6
; GFX9-NEXT: v_mov_b32_e32 v6, s57
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v20
-; GFX9-NEXT: v_mov_b32_e32 v47, v40
+; GFX9-NEXT: v_mov_b32_e32 v57, v40
; GFX9-NEXT: v_mov_b32_e32 v40, v8
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
@@ -123833,22 +123993,21 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_e32 v6, v6, v7
; GFX9-NEXT: v_mov_b32_e32 v7, s45
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v20
-; GFX9-NEXT: v_mov_b32_e32 v43, v9
+; GFX9-NEXT: v_mov_b32_e32 v44, v9
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v9, s15
; GFX9-NEXT: v_or_b32_e32 v7, v7, v8
; GFX9-NEXT: v_mov_b32_e32 v8, s41
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v20
-; GFX9-NEXT: v_mov_b32_e32 v56, v41
-; GFX9-NEXT: v_mov_b32_e32 v41, v10
+; GFX9-NEXT: v_mov_b32_e32 v43, v10
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; GFX9-NEXT: v_mov_b32_e32 v10, s11
; GFX9-NEXT: v_or_b32_e32 v8, v8, v9
; GFX9-NEXT: v_mov_b32_e32 v9, s13
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v20
-; GFX9-NEXT: v_mov_b32_e32 v42, v11
+; GFX9-NEXT: v_mov_b32_e32 v25, v11
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v20
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: v_mov_b32_e32 v11, s7
@@ -123861,22 +124020,21 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v12, v12, v61, s4
-; GFX9-NEXT: v_perm_b32 v14, v52, v14, s4
-; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v32, v15, s4
+; GFX9-NEXT: v_perm_b32 v14, v33, v14, s4
+; GFX9-NEXT: v_perm_b32 v15, v39, v15, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; GFX9-NEXT: v_perm_b32 v16, v53, v16, s4
-; GFX9-NEXT: v_perm_b32 v18, v42, v18, s4
+; GFX9-NEXT: v_perm_b32 v18, v25, v18, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; GFX9-NEXT: v_perm_b32 v17, v35, v17, s4
+; GFX9-NEXT: v_perm_b32 v17, v49, v17, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; GFX9-NEXT: v_perm_b32 v19, v63, v19, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; GFX9-NEXT: v_or_b32_e32 v18, v19, v18
-; GFX9-NEXT: v_perm_b32 v19, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v19, v22, v41, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; GFX9-NEXT: v_perm_b32 v23, v34, v33, s4
-; GFX9-NEXT: v_perm_b32 v24, v41, v43, s4
+; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
+; GFX9-NEXT: v_perm_b32 v25, v47, v56, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v11, s6, v11, v20
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
@@ -123890,81 +124048,83 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_e32 v12, v12, v13
; GFX9-NEXT: v_perm_b32 v13, v27, v29, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; GFX9-NEXT: v_perm_b32 v20, v57, v28, s4
+; GFX9-NEXT: v_perm_b32 v20, v26, v28, s4
; GFX9-NEXT: v_or_b32_e32 v13, v20, v13
-; GFX9-NEXT: v_perm_b32 v20, v55, v48, s4
-; GFX9-NEXT: v_or_b32_e32 v14, v20, v14
-; GFX9-NEXT: v_perm_b32 v20, v54, v39, s4
+; GFX9-NEXT: v_perm_b32 v20, v52, v48, s4
+; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; GFX9-NEXT: v_or_b32_e32 v14, v14, v20
+; GFX9-NEXT: v_perm_b32 v20, v32, v54, s4
; GFX9-NEXT: v_or_b32_e32 v15, v20, v15
; GFX9-NEXT: v_perm_b32 v20, v36, v50, s4
; GFX9-NEXT: v_or_b32_e32 v16, v20, v16
-; GFX9-NEXT: v_perm_b32 v20, v49, v38, s4
+; GFX9-NEXT: v_perm_b32 v20, v35, v38, s4
; GFX9-NEXT: v_or_b32_e32 v17, v20, v17
-; GFX9-NEXT: v_perm_b32 v20, v62, v22, s4
+; GFX9-NEXT: v_perm_b32 v20, v62, v42, s4
; GFX9-NEXT: v_or_b32_e32 v19, v20, v19
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v21, v25, v58, s4
-; GFX9-NEXT: v_perm_b32 v22, v30, v31, s4
-; GFX9-NEXT: v_perm_b32 v25, v45, v46, s4
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v21, v23, v24, s4
+; GFX9-NEXT: v_perm_b32 v23, v34, v55, s4
+; GFX9-NEXT: v_perm_b32 v24, v43, v44, s4
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v26, v20, s4
+; GFX9-NEXT: v_perm_b32 v20, v58, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; GFX9-NEXT: v_or_b32_e32 v20, v21, v20
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v21, v37, v21, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; GFX9-NEXT: v_or_b32_e32 v21, v22, v21
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v22, v51, v22, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; GFX9-NEXT: v_or_b32_e32 v22, v23, v22
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v23, v40, v23, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; GFX9-NEXT: v_or_b32_e32 v23, v24, v23
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v44, v24, s4
+; GFX9-NEXT: v_perm_b32 v24, v45, v24, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; GFX9-NEXT: v_or_b32_e32 v24, v25, v24
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v26, s4
+; GFX9-NEXT: v_perm_b32 v25, v25, v46, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_or_b32_e32 v25, v25, v27
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v26, v27, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_or_b32_e32 v26, v26, v28
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v27, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: v_or_b32_e32 v27, v27, v29
-; GFX9-NEXT: v_perm_b32 v29, v47, v56, s4
-; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
; GFX9-NEXT: v_perm_b32 v28, v59, v28, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_perm_b32 v29, v57, v29, s4
; GFX9-NEXT: v_or_b32_e32 v28, v29, v28
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
@@ -123986,10 +124146,10 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_or_b32_e32 v30, v31, v30
-; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
+; GFX9-NEXT: v_perm_b32 v31, v31, v32, s4
; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -124012,21 +124172,21 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v12, 0xc0c0004
; GFX9-NEXT: s_add_i32 s18, s18, 3
; GFX9-NEXT: v_mov_b32_e32 v1, s19
-; GFX9-NEXT: s_add_i32 s20, s20, 3
-; GFX9-NEXT: v_mov_b32_e32 v2, s21
-; GFX9-NEXT: s_add_i32 s22, s22, 3
-; GFX9-NEXT: v_mov_b32_e32 v3, s23
; GFX9-NEXT: v_perm_b32 v0, s16, v0, v12
-; GFX9-NEXT: s_movk_i32 s4, 0x300
+; GFX9-NEXT: s_movk_i32 s5, 0x300
; GFX9-NEXT: v_perm_b32 v1, s18, v1, v12
-; GFX9-NEXT: v_perm_b32 v2, s20, v2, v12
-; GFX9-NEXT: v_perm_b32 v3, s22, v3, v12
; GFX9-NEXT: v_add_u32_e32 v0, 0x300, v0
-; GFX9-NEXT: v_add_u32_sdwa v1, v1, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v1, v1, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_or_b32_sdwa v1, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX9-NEXT: s_add_i32 s20, s20, 3
+; GFX9-NEXT: v_mov_b32_e32 v1, s21
+; GFX9-NEXT: s_add_i32 s22, s22, 3
+; GFX9-NEXT: v_mov_b32_e32 v2, s23
+; GFX9-NEXT: v_perm_b32 v1, s20, v1, v12
+; GFX9-NEXT: v_perm_b32 v2, s22, v2, v12
+; GFX9-NEXT: v_add_u32_e32 v1, 0x300, v1
+; GFX9-NEXT: v_add_u32_sdwa v2, v2, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s24, s24, 3
; GFX9-NEXT: v_mov_b32_e32 v2, s25
; GFX9-NEXT: s_add_i32 s26, s26, 3
@@ -124034,7 +124194,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v12
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v12
; GFX9-NEXT: v_add_u32_e32 v2, 0x300, v2
-; GFX9-NEXT: v_add_u32_sdwa v3, v3, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v3, v3, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v2, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s28, s28, 3
; GFX9-NEXT: v_mov_b32_e32 v3, s29
@@ -124043,7 +124203,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v12
; GFX9-NEXT: v_perm_b32 v4, s76, v4, v12
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
-; GFX9-NEXT: v_add_u32_sdwa v4, v4, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v3, v3, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_mov_b32_e32 v4, s73
@@ -124052,7 +124212,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v4, s74, v4, v12
; GFX9-NEXT: v_perm_b32 v5, s72, v5, v12
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
-; GFX9-NEXT: v_add_u32_sdwa v5, v5, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v4, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s62, s62, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s61
@@ -124061,7 +124221,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v5, s62, v5, v12
; GFX9-NEXT: v_perm_b32 v6, s60, v6, v12
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u32_sdwa v6, v6, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v5, v5, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s58, s58, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s57
@@ -124070,7 +124230,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v6, s58, v6, v12
; GFX9-NEXT: v_perm_b32 v7, s56, v7, v12
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
-; GFX9-NEXT: v_add_u32_sdwa v7, v7, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v6, v6, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s46, s46, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s45
@@ -124079,7 +124239,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v7, s46, v7, v12
; GFX9-NEXT: v_perm_b32 v8, s44, v8, v12
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
-; GFX9-NEXT: v_add_u32_sdwa v8, v8, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v8, v8, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v7, v7, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v8, s41
@@ -124088,7 +124248,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v8, s42, v8, v12
; GFX9-NEXT: v_perm_b32 v9, s40, v9, v12
; GFX9-NEXT: v_add_u32_e32 v8, 0x300, v8
-; GFX9-NEXT: v_add_u32_sdwa v9, v9, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v9, v9, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v8, v8, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s14, s14, 3
; GFX9-NEXT: v_mov_b32_e32 v9, s13
@@ -124097,7 +124257,7 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v9, s14, v9, v12
; GFX9-NEXT: v_perm_b32 v10, s12, v10, v12
; GFX9-NEXT: v_add_u32_e32 v9, 0x300, v9
-; GFX9-NEXT: v_add_u32_sdwa v10, v10, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v10, v10, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v9, v9, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: s_add_i32 s10, s10, 3
; GFX9-NEXT: v_mov_b32_e32 v10, s9
@@ -124106,321 +124266,321 @@ define inreg <16 x double> @bitcast_v128i8_to_v16f64_scalar(<128 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v10, s10, v10, v12
; GFX9-NEXT: v_perm_b32 v11, s8, v11, v12
; GFX9-NEXT: v_add_u32_e32 v10, 0x300, v10
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
; GFX9-NEXT: s_add_i32 s6, s6, 3
-; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
+; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, s6, v11, v12
+; GFX9-NEXT: v_perm_b32 v11, s6, v11, v12
+; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v12, 0x300, v12
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v11
; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v11, v13, v11, s5
-; GFX9-NEXT: v_add_u32_sdwa v11, v11, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v11, v13, v11, s4
+; GFX9-NEXT: v_add_u32_sdwa v11, v11, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:728 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v13, 3, v12
; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:724 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v13, v12, s5
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
+; GFX9-NEXT: v_perm_b32 v12, v13, v12, s4
+; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v12
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v12, v14, v12, s5
-; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v12, v14, v12, s4
+; GFX9-NEXT: v_add_u32_sdwa v12, v12, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v12, v13, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:720 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v14, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v14, v13, s5
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
+; GFX9-NEXT: v_perm_b32 v13, v14, v13, s4
+; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v13
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v13
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v13, v15, v13, s5
-; GFX9-NEXT: v_add_u32_sdwa v13, v13, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v13, v15, v13, s4
+; GFX9-NEXT: v_add_u32_sdwa v13, v13, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v15, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v15, v14, s5
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v15
+; GFX9-NEXT: v_perm_b32 v14, v15, v14, s4
+; GFX9-NEXT: v_add_u32_e32 v15, 0x300, v14
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v14
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v14, v16, v14, s5
-; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v16, v14, s4
+; GFX9-NEXT: v_add_u32_sdwa v14, v14, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v16, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v16, v15, s5
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v16
+; GFX9-NEXT: v_perm_b32 v15, v16, v15, s4
+; GFX9-NEXT: v_add_u32_e32 v16, 0x300, v15
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v15
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v15, v17, v15, s5
-; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v15, v17, v15, s4
+; GFX9-NEXT: v_add_u32_sdwa v15, v15, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v15, v16, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v17, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v17, v16, s5
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v17
+; GFX9-NEXT: v_perm_b32 v16, v17, v16, s4
+; GFX9-NEXT: v_add_u32_e32 v17, 0x300, v16
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v16
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v18, v16, s5
-; GFX9-NEXT: v_add_u32_sdwa v16, v16, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v16, v18, v16, s4
+; GFX9-NEXT: v_add_u32_sdwa v16, v16, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v16, v17, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v18, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v18, v17, s5
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v18
+; GFX9-NEXT: v_perm_b32 v17, v18, v17, s4
+; GFX9-NEXT: v_add_u32_e32 v18, 0x300, v17
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v17
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v17, v19, v17, s5
-; GFX9-NEXT: v_add_u32_sdwa v17, v17, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v17, v19, v17, s4
+; GFX9-NEXT: v_add_u32_sdwa v17, v17, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v17, v18, v17 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v19, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v19, v18, s5
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v19
+; GFX9-NEXT: v_perm_b32 v18, v19, v18, s4
+; GFX9-NEXT: v_add_u32_e32 v19, 0x300, v18
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v18
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v18, v20, v18, s5
-; GFX9-NEXT: v_add_u32_sdwa v18, v18, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v18, v20, v18, s4
+; GFX9-NEXT: v_add_u32_sdwa v18, v18, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v18, v19, v18 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v20, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v20, v19, s5
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v20
+; GFX9-NEXT: v_perm_b32 v19, v20, v19, s4
+; GFX9-NEXT: v_add_u32_e32 v20, 0x300, v19
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v19
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v19, v21, v19, s5
-; GFX9-NEXT: v_add_u32_sdwa v19, v19, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v19, v21, v19, s4
+; GFX9-NEXT: v_add_u32_sdwa v19, v19, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v19, v20, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v21, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v21, v20, s5
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v21
+; GFX9-NEXT: v_perm_b32 v20, v21, v20, s4
+; GFX9-NEXT: v_add_u32_e32 v21, 0x300, v20
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v20
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v20, v22, v20, s5
-; GFX9-NEXT: v_add_u32_sdwa v20, v20, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v20, v22, v20, s4
+; GFX9-NEXT: v_add_u32_sdwa v20, v20, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v20, v21, v20 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v22, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v22, v21, s5
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v22
+; GFX9-NEXT: v_perm_b32 v21, v22, v21, s4
+; GFX9-NEXT: v_add_u32_e32 v22, 0x300, v21
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v21
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v21, v23, v21, s5
-; GFX9-NEXT: v_add_u32_sdwa v21, v21, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v21, v23, v21, s4
+; GFX9-NEXT: v_add_u32_sdwa v21, v21, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v21, v22, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v23, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v23, v22, s5
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v23
+; GFX9-NEXT: v_perm_b32 v22, v23, v22, s4
+; GFX9-NEXT: v_add_u32_e32 v23, 0x300, v22
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v22
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v22, v24, v22, s5
-; GFX9-NEXT: v_add_u32_sdwa v22, v22, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v22, v24, v22, s4
+; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v22, v23, v22 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v24, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v24, v23, s5
+; GFX9-NEXT: v_perm_b32 v23, v24, v23, s4
+; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v23
; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v24, 0x300, v24
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v23
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v23, v25, v23, s5
-; GFX9-NEXT: v_add_u32_sdwa v23, v23, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v23, v25, v23, s4
+; GFX9-NEXT: v_add_u32_sdwa v23, v23, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v23, v24, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v25, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v25, v24, s5
+; GFX9-NEXT: v_perm_b32 v24, v25, v24, s4
+; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v24
; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u32_e32 v25, 0x300, v25
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v24
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v24, v26, v24, s5
-; GFX9-NEXT: v_add_u32_sdwa v24, v24, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v24, v26, v24, s4
+; GFX9-NEXT: v_add_u32_sdwa v24, v24, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v24, v25, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v26, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v26, v25, s5
+; GFX9-NEXT: v_perm_b32 v25, v26, v25, s4
; GFX9-NEXT: v_add_u32_e32 v26, 0x300, v25
; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v25
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v25, v27, v25, s5
-; GFX9-NEXT: v_add_u32_sdwa v25, v25, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v25, v27, v25, s4
+; GFX9-NEXT: v_add_u32_sdwa v25, v25, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v25, v26, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v27, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v27, v26, s5
+; GFX9-NEXT: v_perm_b32 v26, v27, v26, s4
; GFX9-NEXT: v_add_u32_e32 v27, 0x300, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v26
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v26, v28, v26, s5
-; GFX9-NEXT: v_add_u32_sdwa v26, v26, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v26, v28, v26, s4
+; GFX9-NEXT: v_add_u32_sdwa v26, v26, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v26, v27, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v28, 3, v27
; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v28, v27, s5
+; GFX9-NEXT: v_perm_b32 v27, v28, v27, s4
; GFX9-NEXT: v_add_u32_e32 v28, 0x300, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v27
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v27, v29, v27, s5
-; GFX9-NEXT: v_add_u32_sdwa v27, v27, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v27, v29, v27, s4
+; GFX9-NEXT: v_add_u32_sdwa v27, v27, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v27, v28, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v29, 3, v28
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v29, v28, s5
+; GFX9-NEXT: v_perm_b32 v28, v29, v28, s4
; GFX9-NEXT: v_add_u32_e32 v29, 0x300, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v28
; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v28, v30, v28, s5
-; GFX9-NEXT: v_add_u32_sdwa v28, v28, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v28, v30, v28, s4
+; GFX9-NEXT: v_add_u32_sdwa v28, v28, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v28, v29, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v30, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v30, v29, s5
+; GFX9-NEXT: v_perm_b32 v29, v30, v29, s4
; GFX9-NEXT: v_add_u32_e32 v30, 0x300, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v29
; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v29, v31, v29, s5
-; GFX9-NEXT: v_add_u32_sdwa v29, v29, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v29, v31, v29, s4
+; GFX9-NEXT: v_add_u32_sdwa v29, v29, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v29, v30, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v31, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v31, v30, s5
+; GFX9-NEXT: v_perm_b32 v30, v31, v30, s4
; GFX9-NEXT: v_add_u32_e32 v31, 0x300, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v30
; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v30, v32, v30, s5
-; GFX9-NEXT: v_add_u32_sdwa v30, v30, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v30, v32, v30, s4
+; GFX9-NEXT: v_add_u32_sdwa v30, v30, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v30, v31, v30 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v32, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v32, v31, s5
+; GFX9-NEXT: v_perm_b32 v31, v32, v31, s4
; GFX9-NEXT: v_add_u32_e32 v32, 0x300, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u32_e32 v33, 3, v31
; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v31, v33, v31, s5
-; GFX9-NEXT: v_add_u32_sdwa v31, v31, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v31, v33, v31, s4
+; GFX9-NEXT: v_add_u32_sdwa v31, v31, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v31, v32, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9-NEXT: .LBB75_5: ; %end
; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -126556,100 +126716,100 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: s_cmp_lg_u32 s46, 1
; SI-NEXT: s_cbranch_scc1 .LBB77_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_add_f64 v[22:23], s[4:5], 1.0
-; SI-NEXT: v_add_f64 v[0:1], s[22:23], 1.0
-; SI-NEXT: v_add_f64 v[2:3], s[24:25], 1.0
-; SI-NEXT: v_add_f64 v[14:15], s[14:15], 1.0
-; SI-NEXT: v_add_f64 v[16:17], s[12:13], 1.0
-; SI-NEXT: v_add_f64 v[24:25], s[6:7], 1.0
-; SI-NEXT: v_and_b32_e32 v34, 0xffff0000, v23
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
-; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
+; SI-NEXT: v_add_f64 v[26:27], s[4:5], 1.0
+; SI-NEXT: v_add_f64 v[28:29], s[6:7], 1.0
+; SI-NEXT: v_and_b32_e32 v34, 0xffff0000, v27
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v23, 0xffff0000, v22
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
-; SI-NEXT: v_add_f64 v[10:11], s[42:43], 1.0
-; SI-NEXT: v_add_f64 v[18:19], s[10:11], 1.0
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v22, 0xffff0000, v25
-; SI-NEXT: v_and_b32_e32 v43, 0xffff0000, v17
-; SI-NEXT: v_lshlrev_b32_e32 v42, 16, v17
-; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v15
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v15
-; SI-NEXT: v_and_b32_e32 v57, 0xffff0000, v14
-; SI-NEXT: v_lshlrev_b32_e32 v56, 16, v14
-; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v2
-; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v2
-; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v1
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v1
-; SI-NEXT: v_add_f64 v[1:2], s[20:21], 1.0
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v25
-; SI-NEXT: v_add_f64 v[4:5], s[26:27], 1.0
-; SI-NEXT: v_add_f64 v[12:13], s[40:41], 1.0
-; SI-NEXT: v_add_f64 v[20:21], s[8:9], 1.0
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v22, 0xffff0000, v24
-; SI-NEXT: v_and_b32_e32 v55, 0xffff0000, v19
-; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v19
-; SI-NEXT: v_and_b32_e32 v41, 0xffff0000, v18
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v18
-; SI-NEXT: v_and_b32_e32 v38, 0xffff0000, v11
-; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v11
-; SI-NEXT: v_and_b32_e32 v33, 0xffff0000, v10
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v10
-; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v3
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v3
-; SI-NEXT: v_and_b32_e32 v11, 0xffff0000, v2
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v2
-; SI-NEXT: v_add_f64 v[2:3], s[18:19], 1.0
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_add_f64 v[0:1], s[18:19], 1.0
+; SI-NEXT: v_add_f64 v[2:3], s[20:21], 1.0
+; SI-NEXT: v_add_f64 v[6:7], s[24:25], 1.0
+; SI-NEXT: v_add_f64 v[8:9], s[26:27], 1.0
+; SI-NEXT: v_add_f64 v[16:17], s[40:41], 1.0
+; SI-NEXT: v_add_f64 v[22:23], s[10:11], 1.0
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v26, 0xffff0000, v29
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v24
-; SI-NEXT: v_add_f64 v[6:7], s[28:29], 1.0
-; SI-NEXT: v_add_f64 v[8:9], s[44:45], 1.0
-; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: v_and_b32_e32 v51, 0xffff0000, v21
-; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v21
-; SI-NEXT: v_and_b32_e32 v53, 0xffff0000, v20
-; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v20
-; SI-NEXT: v_and_b32_e32 v59, 0xffff0000, v13
-; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v13
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v29
+; SI-NEXT: v_add_f64 v[4:5], s[22:23], 1.0
+; SI-NEXT: v_add_f64 v[10:11], s[28:29], 1.0
+; SI-NEXT: v_add_f64 v[12:13], s[44:45], 1.0
+; SI-NEXT: v_add_f64 v[14:15], s[42:43], 1.0
+; SI-NEXT: v_add_f64 v[18:19], s[14:15], 1.0
+; SI-NEXT: v_add_f64 v[20:21], s[12:13], 1.0
+; SI-NEXT: v_add_f64 v[24:25], s[8:9], 1.0
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v26, 0xffff0000, v28
+; SI-NEXT: v_and_b32_e32 v55, 0xffff0000, v23
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v23
+; SI-NEXT: v_and_b32_e32 v41, 0xffff0000, v22
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v22
+; SI-NEXT: v_and_b32_e32 v59, 0xffff0000, v17
+; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v17
+; SI-NEXT: v_and_b32_e32 v23, 0xffff0000, v9
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v9
+; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v6
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v6
+; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v2
+; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v2
+; SI-NEXT: v_and_b32_e32 v48, 0xffff0000, v1
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v1
+; SI-NEXT: v_add_f64 v[1:2], s[16:17], 1.0
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v28
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v51, 0xffff0000, v25
+; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v25
+; SI-NEXT: v_and_b32_e32 v53, 0xffff0000, v24
+; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v24
+; SI-NEXT: v_and_b32_e32 v43, 0xffff0000, v21
+; SI-NEXT: v_lshlrev_b32_e32 v42, 16, v21
+; SI-NEXT: v_and_b32_e32 v45, 0xffff0000, v20
+; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v20
+; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v19
+; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v19
+; SI-NEXT: v_and_b32_e32 v57, 0xffff0000, v18
+; SI-NEXT: v_lshlrev_b32_e32 v56, 16, v18
+; SI-NEXT: v_and_b32_e32 v61, 0xffff0000, v16
+; SI-NEXT: v_lshlrev_b32_e32 v60, 16, v16
+; SI-NEXT: v_and_b32_e32 v38, 0xffff0000, v15
+; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v15
+; SI-NEXT: v_and_b32_e32 v33, 0xffff0000, v14
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v14
+; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v13
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v13
+; SI-NEXT: v_and_b32_e32 v29, 0xffff0000, v12
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v12
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_and_b32_e32 v23, 0xffff0000, v5
+; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v11
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v5
-; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v4
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v4
-; SI-NEXT: v_and_b32_e32 v39, 0xffff0000, v1
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v1
-; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v2
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v2
-; SI-NEXT: v_add_f64 v[1:2], s[16:17], 1.0
-; SI-NEXT: v_and_b32_e32 v45, 0xffff0000, v16
-; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v16
-; SI-NEXT: v_and_b32_e32 v61, 0xffff0000, v12
-; SI-NEXT: v_lshlrev_b32_e32 v60, 16, v12
-; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v9
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v9
-; SI-NEXT: v_and_b32_e32 v29, 0xffff0000, v8
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v8
-; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v7
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v7
-; SI-NEXT: v_and_b32_e32 v25, 0xffff0000, v6
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v6
-; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v0
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v0
-; SI-NEXT: v_and_b32_e32 v48, 0xffff0000, v3
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v3
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v11
+; SI-NEXT: v_and_b32_e32 v25, 0xffff0000, v10
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v10
+; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v8
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v8
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v7
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v7
+; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v5
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v5
+; SI-NEXT: v_and_b32_e32 v13, 0xffff0000, v4
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v4
+; SI-NEXT: v_and_b32_e32 v11, 0xffff0000, v3
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v3
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v0
; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v2
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v1
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v1
+; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v1
; SI-NEXT: v_mov_b32_e32 v1, v34
; SI-NEXT: s_branch .LBB77_6
; SI-NEXT: .LBB77_5:
@@ -126720,19 +126880,19 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: v_mov_b32_e32 v49, s88
; SI-NEXT: v_mov_b32_e32 v15, s79
; SI-NEXT: v_mov_b32_e32 v14, s78
-; SI-NEXT: v_mov_b32_e32 v12, s77
-; SI-NEXT: v_mov_b32_e32 v36, s76
+; SI-NEXT: v_mov_b32_e32 v13, s77
+; SI-NEXT: v_mov_b32_e32 v12, s76
; SI-NEXT: v_mov_b32_e32 v11, s75
; SI-NEXT: v_mov_b32_e32 v10, s74
-; SI-NEXT: v_mov_b32_e32 v39, s73
-; SI-NEXT: v_mov_b32_e32 v13, s72
+; SI-NEXT: v_mov_b32_e32 v9, s73
+; SI-NEXT: v_mov_b32_e32 v39, s72
; SI-NEXT: v_mov_b32_e32 v48, s63
; SI-NEXT: v_mov_b32_e32 v6, s62
; SI-NEXT: v_mov_b32_e32 v5, s61
; SI-NEXT: v_mov_b32_e32 v4, s60
; SI-NEXT: v_mov_b32_e32 v3, s59
; SI-NEXT: v_mov_b32_e32 v2, s58
-; SI-NEXT: v_mov_b32_e32 v9, s4
+; SI-NEXT: v_mov_b32_e32 v36, s4
; SI-NEXT: .LBB77_6: ; %end
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v0
; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v0
@@ -126742,7 +126902,7 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v0
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v48
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v0
-; SI-NEXT: v_mul_f32_e32 v0, 1.0, v39
+; SI-NEXT: v_mul_f32_e32 v0, 1.0, v9
; SI-NEXT: v_mul_f32_e32 v7, 1.0, v4
; SI-NEXT: v_mul_f32_e32 v4, 1.0, v6
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v0
@@ -126751,11 +126911,10 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: v_mul_f32_e32 v10, 1.0, v10
; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: v_mul_f32_e32 v0, 1.0, v12
-; SI-NEXT: v_mul_f32_e32 v34, 1.0, v9
+; SI-NEXT: v_mul_f32_e32 v0, 1.0, v13
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v0
-; SI-NEXT: v_mul_f32_e32 v9, 1.0, v36
+; SI-NEXT: v_mul_f32_e32 v9, 1.0, v12
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v15
; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
@@ -126779,13 +126938,9 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v31
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v0
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v33
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: v_mul_f32_e32 v11, 1.0, v18
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v0
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v38
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mul_f32_e32 v5, 1.0, v13
; SI-NEXT: v_mul_f32_e32 v13, 1.0, v20
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v0
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v61
@@ -126810,12 +126965,18 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: v_mul_f32_e32 v9, 1.0, v49
+; SI-NEXT: v_mul_f32_e32 v34, 1.0, v36
; SI-NEXT: v_mul_f32_e32 v2, 1.0, v2
; SI-NEXT: v_lshr_b64 v[33:34], v[34:35], 16
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: v_lshr_b64 v[34:35], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[2:3], v[7:8], 16
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e32 v5, 1.0, v39
; SI-NEXT: v_mul_f32_e32 v19, 1.0, v37
; SI-NEXT: v_mul_f32_e32 v22, 1.0, v22
; SI-NEXT: v_mul_f32_e32 v24, 1.0, v24
@@ -126869,7 +127030,7 @@ define inreg <64 x bfloat> @bitcast_v16f64_to_v64bf16_scalar(<16 x double> inreg
; SI-NEXT: v_readlane_b32 s36, v63, 2
; SI-NEXT: v_readlane_b32 s35, v63, 1
; SI-NEXT: v_readlane_b32 s34, v63, 0
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v0
; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
@@ -134111,34 +134272,34 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Spill
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:4
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: ; implicit-def: $vgpr46
+; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: ; implicit-def: $vgpr43
+; SI-NEXT: ; implicit-def: $vgpr41
; SI-NEXT: ; implicit-def: $vgpr61
-; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr60
; SI-NEXT: ; implicit-def: $vgpr52
; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr58
; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr48
-; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr45
+; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr45
; SI-NEXT: ; implicit-def: $vgpr37
-; SI-NEXT: ; implicit-def: $vgpr42
+; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr41
+; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr55
+; SI-NEXT: ; implicit-def: $vgpr40
; SI-NEXT: ; implicit-def: $vgpr34
-; SI-NEXT: ; implicit-def: $vgpr54
+; SI-NEXT: ; implicit-def: $vgpr55
; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr51
@@ -134162,21 +134323,21 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v49, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v50, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v52, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v40, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v43, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v46, v3, v2, 16
-; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: v_alignbit_b32 v57, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v54, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v41, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v44, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v47, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v31
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
-; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v25
-; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v23
-; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v27
+; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v25
+; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v21
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v15
+; SI-NEXT: s_waitcnt expcnt(6)
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v13
; SI-NEXT: s_waitcnt expcnt(5)
; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v11
; SI-NEXT: s_waitcnt expcnt(4)
@@ -134222,21 +134383,21 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v49, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v50, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v52, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v40, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v43, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v46, v3, v2, 16
-; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: v_alignbit_b32 v57, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v54, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v41, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v44, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v47, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v31
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
-; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v25
-; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v23
-; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v27
+; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v25
+; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v21
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v15
+; SI-NEXT: s_waitcnt expcnt(6)
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v13
; SI-NEXT: s_waitcnt expcnt(5)
; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v11
; SI-NEXT: s_waitcnt expcnt(4)
@@ -134251,27 +134412,23 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v1
; SI-NEXT: .LBB80_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v57
+; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v47
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v46
+; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v44
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v43, 16, v43
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v41
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v40
-; SI-NEXT: v_or_b32_e32 v0, v0, v57
+; SI-NEXT: v_or_b32_e32 v0, v0, v47
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v63
-; SI-NEXT: v_or_b32_e32 v2, v2, v46
+; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v63
+; SI-NEXT: v_or_b32_e32 v2, v2, v44
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v62
-; SI-NEXT: v_or_b32_e32 v4, v4, v43
+; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v62
+; SI-NEXT: v_or_b32_e32 v4, v4, v41
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v43, 16, v61
-; SI-NEXT: v_or_b32_e32 v6, v6, v40
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v60
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v61
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v54
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v52
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
@@ -134288,26 +134445,31 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v36
-; SI-NEXT: v_or_b32_e32 v1, v1, v57
-; SI-NEXT: v_or_b32_e32 v3, v3, v46
-; SI-NEXT: v_or_b32_e32 v5, v5, v43
-; SI-NEXT: v_or_b32_e32 v7, v7, v40
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_or_b32_e32 v1, v1, v47
+; SI-NEXT: v_or_b32_e32 v3, v3, v44
+; SI-NEXT: v_or_b32_e32 v5, v5, v41
+; SI-NEXT: v_or_b32_e32 v6, v6, v54
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v60
; SI-NEXT: v_or_b32_e32 v8, v8, v52
; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v59
; SI-NEXT: v_or_b32_e32 v10, v10, v50
; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v58
; SI-NEXT: v_or_b32_e32 v12, v12, v49
-; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v56
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v57
; SI-NEXT: v_or_b32_e32 v14, v14, v48
-; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v47
+; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v56
; SI-NEXT: v_or_b32_e32 v16, v16, v39
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v45
+; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v46
; SI-NEXT: v_or_b32_e32 v18, v18, v38
-; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v44
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v45
; SI-NEXT: v_or_b32_e32 v20, v20, v37
-; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v42
+; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v43
; SI-NEXT: v_or_b32_e32 v22, v22, v36
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v41
+; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v42
+; SI-NEXT: v_or_b32_e32 v24, v24, v35
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v40
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -134325,14 +134487,13 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
@@ -134341,12 +134502,10 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_and_b32_e32 v21, 0xffff, v21
; SI-NEXT: v_and_b32_e32 v23, 0xffff, v23
-; SI-NEXT: v_or_b32_e32 v24, v24, v35
; SI-NEXT: v_and_b32_e32 v25, 0xffff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v55
; SI-NEXT: v_or_b32_e32 v26, v26, v34
; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
-; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v54
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v55
; SI-NEXT: v_or_b32_e32 v28, v28, v33
; SI-NEXT: v_and_b32_e32 v29, 0xffff, v29
; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v53
@@ -134354,6 +134513,7 @@ define <64 x half> @bitcast_v16f64_to_v64f16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_and_b32_e32 v31, 0xffff, v31
; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v51
+; SI-NEXT: v_or_b32_e32 v7, v7, v54
; SI-NEXT: v_or_b32_e32 v9, v9, v52
; SI-NEXT: v_or_b32_e32 v11, v11, v50
; SI-NEXT: v_or_b32_e32 v13, v13, v49
@@ -135279,15 +135439,15 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v26
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v25
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v24
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v23
@@ -135300,12 +135460,12 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v21
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v20
; SI-NEXT: v_mov_b32_e32 v52, v18
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v19
; SI-NEXT: v_mov_b32_e32 v53, v17
@@ -135345,18 +135505,18 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v42
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v43
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v44
+; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v43
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v44
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v45
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v46
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v47
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v56
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v57
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v59
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v46
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v47
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v56
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v57
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v58
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v59
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v60
; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v61
-; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v62
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v62
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
@@ -135381,7 +135541,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v62
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v63
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v36
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v61
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v50
; SI-NEXT: v_or_b32_e32 v0, v0, v1
@@ -135390,31 +135550,31 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v32
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v59
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v49
; SI-NEXT: v_or_b32_e32 v3, v3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v58
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v35
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v48
; SI-NEXT: v_or_b32_e32 v5, v5, v6
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v56
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v39
; SI-NEXT: v_or_b32_e32 v6, v6, v7
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v33
; SI-NEXT: v_or_b32_e32 v7, v7, v8
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v46
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v38
; SI-NEXT: v_or_b32_e32 v8, v8, v9
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v45
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v34
; SI-NEXT: v_or_b32_e32 v9, v9, v10
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v44
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v37
; SI-NEXT: v_or_b32_e32 v10, v10, v11
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v43
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v63
; SI-NEXT: v_or_b32_e32 v11, v11, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v42
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v51
@@ -135472,25 +135632,25 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr42
-; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; kill: killed $vgpr51
; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
@@ -135518,7 +135678,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_or_b32_e32 v18, v18, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: s_waitcnt vmcnt(11)
@@ -135552,7 +135712,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_or_b32_e32 v19, v19, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v20, 0xffff, v20
; SI-NEXT: v_or_b32_e32 v20, v20, v21
@@ -135628,7 +135788,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: s_cbranch_execz .LBB82_4
; SI-NEXT: ; %bb.3: ; %cmp.true
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, v63
+; SI-NEXT: v_cvt_f32_f16_e32 v0, v36
; SI-NEXT: v_cvt_f32_f16_e32 v2, v50
; SI-NEXT: v_cvt_f32_f16_e32 v1, v62
; SI-NEXT: v_cvt_f32_f16_e32 v3, v61
@@ -135656,7 +135816,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v3, v36
+; SI-NEXT: v_cvt_f32_f16_e32 v3, v49
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_cvt_f32_f16_e32 v6, v57
@@ -135667,7 +135827,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, v49
+; SI-NEXT: v_cvt_f32_f16_e32 v4, v35
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_cvt_f32_f16_e32 v8, v47
@@ -135678,7 +135838,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v5, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v5, v33
+; SI-NEXT: v_cvt_f32_f16_e32 v5, v48
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_cvt_f32_f16_e32 v10, v45
@@ -135689,7 +135849,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, v48
+; SI-NEXT: v_cvt_f32_f16_e32 v6, v39
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_cvt_f32_f16_e32 v12, v43
@@ -135700,7 +135860,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v7, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v7, v35
+; SI-NEXT: v_cvt_f32_f16_e32 v7, v33
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f32_f16_e32 v14, v41
@@ -135711,7 +135871,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v8, v39
+; SI-NEXT: v_cvt_f32_f16_e32 v8, v38
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f32_f16_e32 v16, v55
@@ -135733,23 +135893,23 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v10, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v10, v38
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: v_cvt_f32_f16_e32 v10, v37
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v11, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v11, v37
+; SI-NEXT: v_cvt_f32_f16_e32 v11, v63
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -135878,7 +136038,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_cvt_f32_f16_e32 v25, v25
; SI-NEXT: s_waitcnt vmcnt(2)
@@ -135894,7 +136054,7 @@ define <16 x double> @bitcast_v64f16_to_v16f64(<64 x half> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -137368,34 +137528,34 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Spill
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:4
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: ; implicit-def: $vgpr46
+; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: ; implicit-def: $vgpr43
+; SI-NEXT: ; implicit-def: $vgpr41
; SI-NEXT: ; implicit-def: $vgpr61
-; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr60
; SI-NEXT: ; implicit-def: $vgpr52
; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr58
; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr48
-; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr45
+; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr45
; SI-NEXT: ; implicit-def: $vgpr37
-; SI-NEXT: ; implicit-def: $vgpr42
+; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr41
+; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr55
+; SI-NEXT: ; implicit-def: $vgpr40
; SI-NEXT: ; implicit-def: $vgpr34
-; SI-NEXT: ; implicit-def: $vgpr54
+; SI-NEXT: ; implicit-def: $vgpr55
; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr51
@@ -137419,21 +137579,21 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v49, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v50, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v52, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v40, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v43, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v46, v3, v2, 16
-; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: v_alignbit_b32 v57, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v54, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v41, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v44, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v47, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v31
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
-; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v25
-; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v23
-; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v27
+; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v25
+; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v21
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v15
+; SI-NEXT: s_waitcnt expcnt(6)
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v13
; SI-NEXT: s_waitcnt expcnt(5)
; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v11
; SI-NEXT: s_waitcnt expcnt(4)
@@ -137479,21 +137639,21 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v49, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v50, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v52, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v40, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v43, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v46, v3, v2, 16
-; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: v_alignbit_b32 v57, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v54, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v41, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v44, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v47, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v31
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
-; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v25
-; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v23
-; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v27
+; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v25
+; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v21
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v15
+; SI-NEXT: s_waitcnt expcnt(6)
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v13
; SI-NEXT: s_waitcnt expcnt(5)
; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v11
; SI-NEXT: s_waitcnt expcnt(4)
@@ -137508,27 +137668,23 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v1
; SI-NEXT: .LBB84_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(6)
-; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v57
+; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v47
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v46
+; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v44
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v43, 16, v43
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v41
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v40
-; SI-NEXT: v_or_b32_e32 v0, v0, v57
+; SI-NEXT: v_or_b32_e32 v0, v0, v47
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v63
-; SI-NEXT: v_or_b32_e32 v2, v2, v46
+; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v63
+; SI-NEXT: v_or_b32_e32 v2, v2, v44
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v62
-; SI-NEXT: v_or_b32_e32 v4, v4, v43
+; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v62
+; SI-NEXT: v_or_b32_e32 v4, v4, v41
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v43, 16, v61
-; SI-NEXT: v_or_b32_e32 v6, v6, v40
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v60
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v61
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v54
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v52
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
@@ -137545,26 +137701,31 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v36
-; SI-NEXT: v_or_b32_e32 v1, v1, v57
-; SI-NEXT: v_or_b32_e32 v3, v3, v46
-; SI-NEXT: v_or_b32_e32 v5, v5, v43
-; SI-NEXT: v_or_b32_e32 v7, v7, v40
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_or_b32_e32 v1, v1, v47
+; SI-NEXT: v_or_b32_e32 v3, v3, v44
+; SI-NEXT: v_or_b32_e32 v5, v5, v41
+; SI-NEXT: v_or_b32_e32 v6, v6, v54
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v60
; SI-NEXT: v_or_b32_e32 v8, v8, v52
; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v59
; SI-NEXT: v_or_b32_e32 v10, v10, v50
; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v58
; SI-NEXT: v_or_b32_e32 v12, v12, v49
-; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v56
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v57
; SI-NEXT: v_or_b32_e32 v14, v14, v48
-; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v47
+; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v56
; SI-NEXT: v_or_b32_e32 v16, v16, v39
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v45
+; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v46
; SI-NEXT: v_or_b32_e32 v18, v18, v38
-; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v44
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v45
; SI-NEXT: v_or_b32_e32 v20, v20, v37
-; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v42
+; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v43
; SI-NEXT: v_or_b32_e32 v22, v22, v36
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v41
+; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v42
+; SI-NEXT: v_or_b32_e32 v24, v24, v35
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v40
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -137582,14 +137743,13 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
@@ -137598,12 +137758,10 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_and_b32_e32 v21, 0xffff, v21
; SI-NEXT: v_and_b32_e32 v23, 0xffff, v23
-; SI-NEXT: v_or_b32_e32 v24, v24, v35
; SI-NEXT: v_and_b32_e32 v25, 0xffff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v55
; SI-NEXT: v_or_b32_e32 v26, v26, v34
; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
-; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v54
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v55
; SI-NEXT: v_or_b32_e32 v28, v28, v33
; SI-NEXT: v_and_b32_e32 v29, 0xffff, v29
; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v53
@@ -137611,6 +137769,7 @@ define <64 x i16> @bitcast_v16f64_to_v64i16(<16 x double> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_and_b32_e32 v31, 0xffff, v31
; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v51
+; SI-NEXT: v_or_b32_e32 v7, v7, v54
; SI-NEXT: v_or_b32_e32 v9, v9, v52
; SI-NEXT: v_or_b32_e32 v11, v11, v50
; SI-NEXT: v_or_b32_e32 v13, v13, v49
@@ -209332,8 +209491,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v34, vcc, 0x7fff, v34
; VI-NEXT: v_or_b32_e32 v35, 0x400000, v17
; VI-NEXT: v_cmp_u_f32_e32 vcc, v17, v17
-; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; VI-NEXT: v_cndmask_b32_e32 v32, v34, v35, vcc
+; VI-NEXT: v_cndmask_b32_e32 v17, v34, v35, vcc
; VI-NEXT: v_lshlrev_b32_e32 v34, 16, v18
; VI-NEXT: v_add_f32_e32 v34, 0x40c00000, v34
; VI-NEXT: v_bfe_u32 v35, v34, 16, 1
@@ -209542,7 +209700,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v40, 0x400000, v30
; VI-NEXT: v_cmp_u_f32_e32 vcc, v30, v30
; VI-NEXT: v_cndmask_b32_e32 v30, v55, v40, vcc
-; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_lshlrev_b32_e32 v55, 16, v31
; VI-NEXT: v_add_f32_e32 v55, 0x40c00000, v55
; VI-NEXT: v_bfe_u32 v40, v55, 16, 1
@@ -209574,7 +209732,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v41, vcc, 0x7fff, v41
; VI-NEXT: v_or_b32_e32 v42, 0x400000, v0
; VI-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
-; VI-NEXT: v_cndmask_b32_e32 v17, v41, v42, vcc
+; VI-NEXT: v_cndmask_b32_e32 v0, v41, v42, vcc
; VI-NEXT: v_lshlrev_b32_e32 v41, 16, v1
; VI-NEXT: v_add_f32_e32 v41, 0x40c00000, v41
; VI-NEXT: v_bfe_u32 v42, v41, 16, 1
@@ -209590,8 +209748,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v42, vcc, 0x7fff, v42
; VI-NEXT: v_or_b32_e32 v43, 0x400000, v1
; VI-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; VI-NEXT: v_cndmask_b32_e32 v16, v42, v43, vcc
+; VI-NEXT: v_cndmask_b32_e32 v1, v42, v43, vcc
; VI-NEXT: v_lshlrev_b32_e32 v42, 16, v2
; VI-NEXT: v_add_f32_e32 v42, 0x40c00000, v42
; VI-NEXT: v_bfe_u32 v43, v42, 16, 1
@@ -209614,274 +209771,273 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v44, vcc, v44, v43
; VI-NEXT: v_add_u32_e32 v44, vcc, 0x7fff, v44
; VI-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
-; VI-NEXT: v_or_b32_e32 v45, 0x400000, v43
; VI-NEXT: v_cmp_u_f32_e32 vcc, v43, v43
+; VI-NEXT: v_or_b32_e32 v43, 0x400000, v43
; VI-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
-; VI-NEXT: v_cndmask_b32_e32 v43, v44, v45, vcc
+; VI-NEXT: v_cndmask_b32_e32 v43, v44, v43, vcc
; VI-NEXT: v_bfe_u32 v44, v3, 16, 1
; VI-NEXT: v_add_u32_e32 v44, vcc, v44, v3
; VI-NEXT: v_add_u32_e32 v44, vcc, 0x7fff, v44
-; VI-NEXT: v_or_b32_e32 v45, 0x400000, v3
; VI-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
-; VI-NEXT: v_cndmask_b32_e32 v3, v44, v45, vcc
+; VI-NEXT: v_or_b32_e32 v3, 0x400000, v3
+; VI-NEXT: v_cndmask_b32_e32 v3, v44, v3, vcc
; VI-NEXT: v_lshlrev_b32_e32 v44, 16, v4
; VI-NEXT: v_add_f32_e32 v44, 0x40c00000, v44
; VI-NEXT: v_bfe_u32 v45, v44, 16, 1
; VI-NEXT: v_add_u32_e32 v45, vcc, v45, v44
; VI-NEXT: v_add_u32_e32 v45, vcc, 0x7fff, v45
; VI-NEXT: v_and_b32_e32 v4, 0xffff0000, v4
-; VI-NEXT: v_or_b32_e32 v46, 0x400000, v44
; VI-NEXT: v_cmp_u_f32_e32 vcc, v44, v44
+; VI-NEXT: v_or_b32_e32 v44, 0x400000, v44
; VI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
-; VI-NEXT: v_cndmask_b32_e32 v44, v45, v46, vcc
+; VI-NEXT: v_cndmask_b32_e32 v44, v45, v44, vcc
; VI-NEXT: v_bfe_u32 v45, v4, 16, 1
; VI-NEXT: v_add_u32_e32 v45, vcc, v45, v4
; VI-NEXT: v_add_u32_e32 v45, vcc, 0x7fff, v45
-; VI-NEXT: v_or_b32_e32 v46, 0x400000, v4
; VI-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
-; VI-NEXT: v_cndmask_b32_e32 v4, v45, v46, vcc
+; VI-NEXT: v_or_b32_e32 v4, 0x400000, v4
+; VI-NEXT: v_cndmask_b32_e32 v4, v45, v4, vcc
; VI-NEXT: v_lshlrev_b32_e32 v45, 16, v5
; VI-NEXT: v_add_f32_e32 v45, 0x40c00000, v45
; VI-NEXT: v_bfe_u32 v46, v45, 16, 1
; VI-NEXT: v_add_u32_e32 v46, vcc, v46, v45
; VI-NEXT: v_add_u32_e32 v46, vcc, 0x7fff, v46
; VI-NEXT: v_and_b32_e32 v5, 0xffff0000, v5
-; VI-NEXT: v_or_b32_e32 v47, 0x400000, v45
; VI-NEXT: v_cmp_u_f32_e32 vcc, v45, v45
+; VI-NEXT: v_or_b32_e32 v45, 0x400000, v45
; VI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
-; VI-NEXT: v_cndmask_b32_e32 v45, v46, v47, vcc
+; VI-NEXT: v_cndmask_b32_e32 v45, v46, v45, vcc
; VI-NEXT: v_bfe_u32 v46, v5, 16, 1
; VI-NEXT: v_add_u32_e32 v46, vcc, v46, v5
; VI-NEXT: v_add_u32_e32 v46, vcc, 0x7fff, v46
-; VI-NEXT: v_or_b32_e32 v47, 0x400000, v5
; VI-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; VI-NEXT: v_cndmask_b32_e32 v5, v46, v47, vcc
+; VI-NEXT: v_or_b32_e32 v5, 0x400000, v5
+; VI-NEXT: v_cndmask_b32_e32 v5, v46, v5, vcc
; VI-NEXT: v_lshlrev_b32_e32 v46, 16, v6
; VI-NEXT: v_add_f32_e32 v46, 0x40c00000, v46
; VI-NEXT: v_bfe_u32 v47, v46, 16, 1
; VI-NEXT: v_add_u32_e32 v47, vcc, v47, v46
; VI-NEXT: v_add_u32_e32 v47, vcc, 0x7fff, v47
; VI-NEXT: v_and_b32_e32 v6, 0xffff0000, v6
-; VI-NEXT: v_or_b32_e32 v56, 0x400000, v46
; VI-NEXT: v_cmp_u_f32_e32 vcc, v46, v46
+; VI-NEXT: v_or_b32_e32 v46, 0x400000, v46
; VI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
-; VI-NEXT: v_cndmask_b32_e32 v46, v47, v56, vcc
+; VI-NEXT: v_cndmask_b32_e32 v46, v47, v46, vcc
; VI-NEXT: v_bfe_u32 v47, v6, 16, 1
; VI-NEXT: v_add_u32_e32 v47, vcc, v47, v6
; VI-NEXT: v_add_u32_e32 v47, vcc, 0x7fff, v47
-; VI-NEXT: v_or_b32_e32 v56, 0x400000, v6
; VI-NEXT: v_cmp_u_f32_e32 vcc, v6, v6
-; VI-NEXT: v_cndmask_b32_e32 v6, v47, v56, vcc
+; VI-NEXT: v_or_b32_e32 v6, 0x400000, v6
+; VI-NEXT: v_cndmask_b32_e32 v6, v47, v6, vcc
; VI-NEXT: v_lshlrev_b32_e32 v47, 16, v7
; VI-NEXT: v_add_f32_e32 v47, 0x40c00000, v47
; VI-NEXT: v_bfe_u32 v56, v47, 16, 1
; VI-NEXT: v_add_u32_e32 v56, vcc, v56, v47
; VI-NEXT: v_add_u32_e32 v56, vcc, 0x7fff, v56
; VI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
-; VI-NEXT: v_or_b32_e32 v57, 0x400000, v47
; VI-NEXT: v_cmp_u_f32_e32 vcc, v47, v47
+; VI-NEXT: v_or_b32_e32 v47, 0x400000, v47
; VI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; VI-NEXT: v_cndmask_b32_e32 v47, v56, v57, vcc
+; VI-NEXT: v_cndmask_b32_e32 v47, v56, v47, vcc
; VI-NEXT: v_bfe_u32 v56, v7, 16, 1
; VI-NEXT: v_add_u32_e32 v56, vcc, v56, v7
; VI-NEXT: v_add_u32_e32 v56, vcc, 0x7fff, v56
-; VI-NEXT: v_or_b32_e32 v57, 0x400000, v7
; VI-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
-; VI-NEXT: v_cndmask_b32_e32 v7, v56, v57, vcc
+; VI-NEXT: v_or_b32_e32 v7, 0x400000, v7
+; VI-NEXT: v_cndmask_b32_e32 v7, v56, v7, vcc
; VI-NEXT: v_lshlrev_b32_e32 v56, 16, v8
; VI-NEXT: v_add_f32_e32 v56, 0x40c00000, v56
; VI-NEXT: v_bfe_u32 v57, v56, 16, 1
; VI-NEXT: v_add_u32_e32 v57, vcc, v57, v56
; VI-NEXT: v_add_u32_e32 v57, vcc, 0x7fff, v57
; VI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; VI-NEXT: v_or_b32_e32 v58, 0x400000, v56
; VI-NEXT: v_cmp_u_f32_e32 vcc, v56, v56
+; VI-NEXT: v_or_b32_e32 v56, 0x400000, v56
; VI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; VI-NEXT: v_cndmask_b32_e32 v56, v57, v58, vcc
+; VI-NEXT: v_cndmask_b32_e32 v56, v57, v56, vcc
; VI-NEXT: v_bfe_u32 v57, v8, 16, 1
; VI-NEXT: v_add_u32_e32 v57, vcc, v57, v8
; VI-NEXT: v_add_u32_e32 v57, vcc, 0x7fff, v57
-; VI-NEXT: v_or_b32_e32 v58, 0x400000, v8
; VI-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; VI-NEXT: v_cndmask_b32_e32 v8, v57, v58, vcc
+; VI-NEXT: v_or_b32_e32 v8, 0x400000, v8
+; VI-NEXT: v_cndmask_b32_e32 v8, v57, v8, vcc
; VI-NEXT: v_lshlrev_b32_e32 v57, 16, v9
; VI-NEXT: v_add_f32_e32 v57, 0x40c00000, v57
; VI-NEXT: v_bfe_u32 v58, v57, 16, 1
; VI-NEXT: v_add_u32_e32 v58, vcc, v58, v57
; VI-NEXT: v_add_u32_e32 v58, vcc, 0x7fff, v58
; VI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; VI-NEXT: v_or_b32_e32 v59, 0x400000, v57
; VI-NEXT: v_cmp_u_f32_e32 vcc, v57, v57
+; VI-NEXT: v_or_b32_e32 v57, 0x400000, v57
; VI-NEXT: v_add_f32_e32 v9, 0x40c00000, v9
-; VI-NEXT: v_cndmask_b32_e32 v57, v58, v59, vcc
+; VI-NEXT: v_cndmask_b32_e32 v57, v58, v57, vcc
; VI-NEXT: v_bfe_u32 v58, v9, 16, 1
; VI-NEXT: v_add_u32_e32 v58, vcc, v58, v9
; VI-NEXT: v_add_u32_e32 v58, vcc, 0x7fff, v58
-; VI-NEXT: v_or_b32_e32 v59, 0x400000, v9
; VI-NEXT: v_cmp_u_f32_e32 vcc, v9, v9
-; VI-NEXT: v_cndmask_b32_e32 v9, v58, v59, vcc
+; VI-NEXT: v_or_b32_e32 v9, 0x400000, v9
+; VI-NEXT: v_cndmask_b32_e32 v9, v58, v9, vcc
; VI-NEXT: v_lshlrev_b32_e32 v58, 16, v10
; VI-NEXT: v_add_f32_e32 v58, 0x40c00000, v58
; VI-NEXT: v_bfe_u32 v59, v58, 16, 1
; VI-NEXT: v_add_u32_e32 v59, vcc, v59, v58
; VI-NEXT: v_add_u32_e32 v59, vcc, 0x7fff, v59
; VI-NEXT: v_and_b32_e32 v10, 0xffff0000, v10
-; VI-NEXT: v_or_b32_e32 v60, 0x400000, v58
; VI-NEXT: v_cmp_u_f32_e32 vcc, v58, v58
+; VI-NEXT: v_or_b32_e32 v58, 0x400000, v58
; VI-NEXT: v_add_f32_e32 v10, 0x40c00000, v10
-; VI-NEXT: v_cndmask_b32_e32 v58, v59, v60, vcc
+; VI-NEXT: v_cndmask_b32_e32 v58, v59, v58, vcc
; VI-NEXT: v_bfe_u32 v59, v10, 16, 1
; VI-NEXT: v_add_u32_e32 v59, vcc, v59, v10
; VI-NEXT: v_add_u32_e32 v59, vcc, 0x7fff, v59
-; VI-NEXT: v_or_b32_e32 v60, 0x400000, v10
; VI-NEXT: v_cmp_u_f32_e32 vcc, v10, v10
-; VI-NEXT: v_cndmask_b32_e32 v10, v59, v60, vcc
+; VI-NEXT: v_or_b32_e32 v10, 0x400000, v10
+; VI-NEXT: v_cndmask_b32_e32 v10, v59, v10, vcc
; VI-NEXT: v_lshlrev_b32_e32 v59, 16, v11
; VI-NEXT: v_add_f32_e32 v59, 0x40c00000, v59
; VI-NEXT: v_bfe_u32 v60, v59, 16, 1
; VI-NEXT: v_add_u32_e32 v60, vcc, v60, v59
; VI-NEXT: v_add_u32_e32 v60, vcc, 0x7fff, v60
; VI-NEXT: v_and_b32_e32 v11, 0xffff0000, v11
-; VI-NEXT: v_or_b32_e32 v61, 0x400000, v59
; VI-NEXT: v_cmp_u_f32_e32 vcc, v59, v59
+; VI-NEXT: v_or_b32_e32 v59, 0x400000, v59
; VI-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
-; VI-NEXT: v_cndmask_b32_e32 v59, v60, v61, vcc
+; VI-NEXT: v_cndmask_b32_e32 v59, v60, v59, vcc
; VI-NEXT: v_bfe_u32 v60, v11, 16, 1
; VI-NEXT: v_add_u32_e32 v60, vcc, v60, v11
; VI-NEXT: v_add_u32_e32 v60, vcc, 0x7fff, v60
-; VI-NEXT: v_or_b32_e32 v61, 0x400000, v11
; VI-NEXT: v_cmp_u_f32_e32 vcc, v11, v11
-; VI-NEXT: v_cndmask_b32_e32 v11, v60, v61, vcc
+; VI-NEXT: v_or_b32_e32 v11, 0x400000, v11
+; VI-NEXT: v_cndmask_b32_e32 v11, v60, v11, vcc
; VI-NEXT: v_lshlrev_b32_e32 v60, 16, v12
; VI-NEXT: v_add_f32_e32 v60, 0x40c00000, v60
; VI-NEXT: v_bfe_u32 v61, v60, 16, 1
; VI-NEXT: v_add_u32_e32 v61, vcc, v61, v60
; VI-NEXT: v_add_u32_e32 v61, vcc, 0x7fff, v61
; VI-NEXT: v_and_b32_e32 v12, 0xffff0000, v12
-; VI-NEXT: v_or_b32_e32 v62, 0x400000, v60
; VI-NEXT: v_cmp_u_f32_e32 vcc, v60, v60
+; VI-NEXT: v_or_b32_e32 v60, 0x400000, v60
; VI-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
-; VI-NEXT: v_cndmask_b32_e32 v60, v61, v62, vcc
+; VI-NEXT: v_cndmask_b32_e32 v60, v61, v60, vcc
; VI-NEXT: v_bfe_u32 v61, v12, 16, 1
; VI-NEXT: v_add_u32_e32 v61, vcc, v61, v12
; VI-NEXT: v_add_u32_e32 v61, vcc, 0x7fff, v61
-; VI-NEXT: v_or_b32_e32 v62, 0x400000, v12
; VI-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
-; VI-NEXT: v_cndmask_b32_e32 v12, v61, v62, vcc
+; VI-NEXT: v_or_b32_e32 v12, 0x400000, v12
+; VI-NEXT: v_cndmask_b32_e32 v12, v61, v12, vcc
; VI-NEXT: v_lshlrev_b32_e32 v61, 16, v13
; VI-NEXT: v_add_f32_e32 v61, 0x40c00000, v61
; VI-NEXT: v_bfe_u32 v62, v61, 16, 1
; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v61
; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
; VI-NEXT: v_and_b32_e32 v13, 0xffff0000, v13
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v61
; VI-NEXT: v_cmp_u_f32_e32 vcc, v61, v61
+; VI-NEXT: v_or_b32_e32 v61, 0x400000, v61
; VI-NEXT: v_add_f32_e32 v13, 0x40c00000, v13
-; VI-NEXT: v_cndmask_b32_e32 v61, v62, v63, vcc
+; VI-NEXT: v_cndmask_b32_e32 v61, v62, v61, vcc
; VI-NEXT: v_bfe_u32 v62, v13, 16, 1
; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v13
; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v13
; VI-NEXT: v_cmp_u_f32_e32 vcc, v13, v13
-; VI-NEXT: v_cndmask_b32_e32 v13, v62, v63, vcc
+; VI-NEXT: v_or_b32_e32 v13, 0x400000, v13
+; VI-NEXT: v_cndmask_b32_e32 v13, v62, v13, vcc
; VI-NEXT: v_lshlrev_b32_e32 v62, 16, v14
; VI-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
; VI-NEXT: v_bfe_u32 v63, v62, 16, 1
; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v62
-; VI-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
-; VI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
-; VI-NEXT: v_or_b32_e32 v0, 0x400000, v62
+; VI-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; VI-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; VI-NEXT: v_bfe_u32 v62, v14, 16, 1
-; VI-NEXT: v_cndmask_b32_e32 v1, v63, v0, vcc
-; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v14
-; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v14
+; VI-NEXT: v_or_b32_e32 v62, 0x400000, v62
+; VI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; VI-NEXT: v_cndmask_b32_e32 v62, v63, v62, vcc
+; VI-NEXT: v_bfe_u32 v63, v14, 16, 1
+; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v14
+; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
; VI-NEXT: v_cmp_u_f32_e32 vcc, v14, v14
-; VI-NEXT: v_cndmask_b32_e32 v14, v62, v63, vcc
-; VI-NEXT: v_lshlrev_b32_e32 v62, 16, v15
-; VI-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
-; VI-NEXT: v_bfe_u32 v63, v62, 16, 1
-; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v62
+; VI-NEXT: v_or_b32_e32 v14, 0x400000, v14
+; VI-NEXT: v_cndmask_b32_e32 v14, v63, v14, vcc
+; VI-NEXT: v_lshlrev_b32_e32 v63, 16, v15
+; VI-NEXT: v_add_f32_e32 v63, 0x40c00000, v63
+; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; VI-NEXT: v_bfe_u32 v32, v63, 16, 1
+; VI-NEXT: v_add_u32_e32 v32, vcc, v32, v63
+; VI-NEXT: v_add_u32_e32 v32, vcc, 0x7fff, v32
; VI-NEXT: v_and_b32_e32 v15, 0xffff0000, v15
-; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
+; VI-NEXT: v_cmp_u_f32_e32 vcc, v63, v63
+; VI-NEXT: v_or_b32_e32 v63, 0x400000, v63
; VI-NEXT: v_add_f32_e32 v15, 0x40c00000, v15
-; VI-NEXT: v_or_b32_e32 v0, 0x400000, v62
-; VI-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; VI-NEXT: v_bfe_u32 v62, v15, 16, 1
-; VI-NEXT: v_cndmask_b32_e32 v0, v63, v0, vcc
-; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v15
-; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v15
+; VI-NEXT: v_cndmask_b32_e32 v32, v32, v63, vcc
+; VI-NEXT: v_bfe_u32 v63, v15, 16, 1
+; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v15
+; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
; VI-NEXT: v_cmp_u_f32_e32 vcc, v15, v15
-; VI-NEXT: v_cndmask_b32_e32 v15, v62, v63, vcc
+; VI-NEXT: v_or_b32_e32 v15, 0x400000, v15
+; VI-NEXT: v_cndmask_b32_e32 v15, v63, v15, vcc
; VI-NEXT: v_lshrrev_b32_e32 v15, 16, v15
-; VI-NEXT: v_alignbit_b32 v15, v15, v0, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v14
-; VI-NEXT: v_alignbit_b32 v14, v0, v1, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v13
-; VI-NEXT: v_alignbit_b32 v13, v0, v61, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v12
-; VI-NEXT: v_alignbit_b32 v12, v0, v60, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v11
-; VI-NEXT: v_alignbit_b32 v11, v0, v59, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v10
-; VI-NEXT: v_alignbit_b32 v10, v0, v58, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v9
-; VI-NEXT: v_alignbit_b32 v9, v0, v57, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v8
-; VI-NEXT: v_alignbit_b32 v8, v0, v56, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v7
-; VI-NEXT: v_alignbit_b32 v7, v0, v47, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v6
-; VI-NEXT: v_alignbit_b32 v6, v0, v46, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v5
-; VI-NEXT: v_alignbit_b32 v5, v0, v45, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v4
-; VI-NEXT: v_alignbit_b32 v4, v0, v44, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v3
-; VI-NEXT: v_alignbit_b32 v3, v0, v43, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v2
-; VI-NEXT: v_alignbit_b32 v2, v0, v42, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v31
-; VI-NEXT: v_alignbit_b32 v31, v16, v55, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v30
-; VI-NEXT: v_alignbit_b32 v30, v16, v54, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v29
-; VI-NEXT: v_alignbit_b32 v29, v16, v53, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v28
-; VI-NEXT: v_alignbit_b32 v28, v16, v52, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v27
-; VI-NEXT: v_alignbit_b32 v27, v16, v51, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v26
-; VI-NEXT: v_alignbit_b32 v26, v16, v50, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v25
-; VI-NEXT: v_alignbit_b32 v25, v16, v49, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v24
-; VI-NEXT: v_alignbit_b32 v24, v16, v48, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v23
-; VI-NEXT: v_alignbit_b32 v23, v16, v39, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v22
-; VI-NEXT: v_alignbit_b32 v22, v16, v38, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v21
-; VI-NEXT: v_alignbit_b32 v21, v16, v37, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v20
-; VI-NEXT: v_alignbit_b32 v20, v16, v36, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v19
-; VI-NEXT: v_alignbit_b32 v19, v16, v35, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v18
-; VI-NEXT: v_alignbit_b32 v18, v16, v34, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v32
-; VI-NEXT: v_alignbit_b32 v1, v0, v41, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v17
-; VI-NEXT: v_alignbit_b32 v17, v16, v33, 16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; VI-NEXT: v_alignbit_b32 v15, v15, v32, 16
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; VI-NEXT: v_alignbit_b32 v0, v0, v40, 16
-; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: v_lshrrev_b32_e32 v14, 16, v14
+; VI-NEXT: v_lshrrev_b32_e32 v13, 16, v13
+; VI-NEXT: v_lshrrev_b32_e32 v12, 16, v12
+; VI-NEXT: v_lshrrev_b32_e32 v11, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v10, 16, v10
+; VI-NEXT: v_lshrrev_b32_e32 v9, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v8, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v7, 16, v7
+; VI-NEXT: v_lshrrev_b32_e32 v6, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v5, 16, v5
+; VI-NEXT: v_lshrrev_b32_e32 v4, 16, v4
+; VI-NEXT: v_lshrrev_b32_e32 v3, 16, v3
+; VI-NEXT: v_lshrrev_b32_e32 v2, 16, v2
+; VI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
+; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v0
+; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v31
+; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v30
+; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v29
+; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v28
+; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v27
+; VI-NEXT: v_lshrrev_b32_e32 v26, 16, v26
+; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v25
+; VI-NEXT: v_lshrrev_b32_e32 v24, 16, v24
+; VI-NEXT: v_lshrrev_b32_e32 v23, 16, v23
+; VI-NEXT: v_lshrrev_b32_e32 v22, 16, v22
+; VI-NEXT: v_lshrrev_b32_e32 v21, 16, v21
+; VI-NEXT: v_lshrrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_lshrrev_b32_e32 v19, 16, v19
+; VI-NEXT: v_lshrrev_b32_e32 v18, 16, v18
+; VI-NEXT: v_lshrrev_b32_e32 v17, 16, v17
; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v16
+; VI-NEXT: v_alignbit_b32 v14, v14, v62, 16
+; VI-NEXT: v_alignbit_b32 v13, v13, v61, 16
+; VI-NEXT: v_alignbit_b32 v12, v12, v60, 16
+; VI-NEXT: v_alignbit_b32 v11, v11, v59, 16
+; VI-NEXT: v_alignbit_b32 v10, v10, v58, 16
+; VI-NEXT: v_alignbit_b32 v9, v9, v57, 16
+; VI-NEXT: v_alignbit_b32 v8, v8, v56, 16
+; VI-NEXT: v_alignbit_b32 v7, v7, v47, 16
+; VI-NEXT: v_alignbit_b32 v6, v6, v46, 16
+; VI-NEXT: v_alignbit_b32 v5, v5, v45, 16
+; VI-NEXT: v_alignbit_b32 v4, v4, v44, 16
+; VI-NEXT: v_alignbit_b32 v3, v3, v43, 16
+; VI-NEXT: v_alignbit_b32 v2, v2, v42, 16
+; VI-NEXT: v_alignbit_b32 v1, v1, v41, 16
+; VI-NEXT: v_alignbit_b32 v0, v0, v40, 16
+; VI-NEXT: v_alignbit_b32 v31, v31, v55, 16
+; VI-NEXT: v_alignbit_b32 v30, v30, v54, 16
+; VI-NEXT: v_alignbit_b32 v29, v29, v53, 16
+; VI-NEXT: v_alignbit_b32 v28, v28, v52, 16
+; VI-NEXT: v_alignbit_b32 v27, v27, v51, 16
+; VI-NEXT: v_alignbit_b32 v26, v26, v50, 16
+; VI-NEXT: v_alignbit_b32 v25, v25, v49, 16
+; VI-NEXT: v_alignbit_b32 v24, v24, v48, 16
+; VI-NEXT: v_alignbit_b32 v23, v23, v39, 16
+; VI-NEXT: v_alignbit_b32 v22, v22, v38, 16
+; VI-NEXT: v_alignbit_b32 v21, v21, v37, 16
+; VI-NEXT: v_alignbit_b32 v20, v20, v36, 16
+; VI-NEXT: v_alignbit_b32 v19, v19, v35, 16
+; VI-NEXT: v_alignbit_b32 v18, v18, v34, 16
+; VI-NEXT: v_alignbit_b32 v17, v17, v33, 16
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_alignbit_b32 v16, v16, v32, 16
; VI-NEXT: .LBB100_2: ; %end
@@ -209961,8 +210117,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_add3_u32 v34, v34, v17, s6
; GFX9-NEXT: v_or_b32_e32 v35, 0x400000, v17
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v17, v17
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; GFX9-NEXT: v_cndmask_b32_e32 v32, v34, v35, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v17, v34, v35, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v34, 16, v18
; GFX9-NEXT: v_add_f32_e32 v34, 0x40c00000, v34
; GFX9-NEXT: v_bfe_u32 v35, v34, 16, 1
@@ -210145,7 +210300,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_or_b32_e32 v40, 0x400000, v30
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v30, v30
; GFX9-NEXT: v_cndmask_b32_e32 v30, v55, v40, vcc
-; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_lshlrev_b32_e32 v55, 16, v31
; GFX9-NEXT: v_add_f32_e32 v55, 0x40c00000, v55
; GFX9-NEXT: v_bfe_u32 v40, v55, 16, 1
@@ -210173,7 +210328,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_add3_u32 v41, v41, v0, s6
; GFX9-NEXT: v_or_b32_e32 v42, 0x400000, v0
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
-; GFX9-NEXT: v_cndmask_b32_e32 v17, v41, v42, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v0, v41, v42, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v41, 16, v1
; GFX9-NEXT: v_add_f32_e32 v41, 0x40c00000, v41
; GFX9-NEXT: v_bfe_u32 v42, v41, 16, 1
@@ -210187,8 +210342,7 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_add3_u32 v42, v42, v1, s6
; GFX9-NEXT: v_or_b32_e32 v43, 0x400000, v1
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; GFX9-NEXT: v_cndmask_b32_e32 v16, v42, v43, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v1, v42, v43, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v42, 16, v2
; GFX9-NEXT: v_add_f32_e32 v42, 0x40c00000, v42
; GFX9-NEXT: v_bfe_u32 v43, v42, 16, 1
@@ -210208,191 +210362,188 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_bfe_u32 v44, v43, 16, 1
; GFX9-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
; GFX9-NEXT: v_add3_u32 v44, v44, v43, s6
-; GFX9-NEXT: v_or_b32_e32 v45, 0x400000, v43
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v43, v43
+; GFX9-NEXT: v_or_b32_e32 v43, 0x400000, v43
; GFX9-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
-; GFX9-NEXT: v_cndmask_b32_e32 v43, v44, v45, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v43, v44, v43, vcc
; GFX9-NEXT: v_bfe_u32 v44, v3, 16, 1
; GFX9-NEXT: v_add3_u32 v44, v44, v3, s6
-; GFX9-NEXT: v_or_b32_e32 v45, 0x400000, v3
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
-; GFX9-NEXT: v_cndmask_b32_e32 v3, v44, v45, vcc
+; GFX9-NEXT: v_or_b32_e32 v3, 0x400000, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v44, v3, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v44, 16, v4
; GFX9-NEXT: v_add_f32_e32 v44, 0x40c00000, v44
; GFX9-NEXT: v_bfe_u32 v45, v44, 16, 1
; GFX9-NEXT: v_and_b32_e32 v4, 0xffff0000, v4
; GFX9-NEXT: v_add3_u32 v45, v45, v44, s6
-; GFX9-NEXT: v_or_b32_e32 v46, 0x400000, v44
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v44, v44
+; GFX9-NEXT: v_or_b32_e32 v44, 0x400000, v44
; GFX9-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
-; GFX9-NEXT: v_cndmask_b32_e32 v44, v45, v46, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v44, v45, v44, vcc
; GFX9-NEXT: v_bfe_u32 v45, v4, 16, 1
; GFX9-NEXT: v_add3_u32 v45, v45, v4, s6
-; GFX9-NEXT: v_or_b32_e32 v46, 0x400000, v4
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
-; GFX9-NEXT: v_cndmask_b32_e32 v4, v45, v46, vcc
+; GFX9-NEXT: v_or_b32_e32 v4, 0x400000, v4
+; GFX9-NEXT: v_cndmask_b32_e32 v4, v45, v4, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v45, 16, v5
; GFX9-NEXT: v_add_f32_e32 v45, 0x40c00000, v45
; GFX9-NEXT: v_bfe_u32 v46, v45, 16, 1
; GFX9-NEXT: v_and_b32_e32 v5, 0xffff0000, v5
; GFX9-NEXT: v_add3_u32 v46, v46, v45, s6
-; GFX9-NEXT: v_or_b32_e32 v47, 0x400000, v45
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v45, v45
+; GFX9-NEXT: v_or_b32_e32 v45, 0x400000, v45
; GFX9-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v45, v46, v47, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v45, v46, v45, vcc
; GFX9-NEXT: v_bfe_u32 v46, v5, 16, 1
; GFX9-NEXT: v_add3_u32 v46, v46, v5, s6
-; GFX9-NEXT: v_or_b32_e32 v47, 0x400000, v5
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v46, v47, vcc
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v5
+; GFX9-NEXT: v_cndmask_b32_e32 v5, v46, v5, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v46, 16, v6
; GFX9-NEXT: v_add_f32_e32 v46, 0x40c00000, v46
; GFX9-NEXT: v_bfe_u32 v47, v46, 16, 1
; GFX9-NEXT: v_and_b32_e32 v6, 0xffff0000, v6
; GFX9-NEXT: v_add3_u32 v47, v47, v46, s6
-; GFX9-NEXT: v_or_b32_e32 v56, 0x400000, v46
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v46, v46
+; GFX9-NEXT: v_or_b32_e32 v46, 0x400000, v46
; GFX9-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
-; GFX9-NEXT: v_cndmask_b32_e32 v46, v47, v56, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v46, v47, v46, vcc
; GFX9-NEXT: v_bfe_u32 v47, v6, 16, 1
; GFX9-NEXT: v_add3_u32 v47, v47, v6, s6
-; GFX9-NEXT: v_or_b32_e32 v56, 0x400000, v6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v6, v6
-; GFX9-NEXT: v_cndmask_b32_e32 v6, v47, v56, vcc
+; GFX9-NEXT: v_or_b32_e32 v6, 0x400000, v6
+; GFX9-NEXT: v_cndmask_b32_e32 v6, v47, v6, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v47, 16, v7
; GFX9-NEXT: v_add_f32_e32 v47, 0x40c00000, v47
; GFX9-NEXT: v_bfe_u32 v56, v47, 16, 1
; GFX9-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
; GFX9-NEXT: v_add3_u32 v56, v56, v47, s6
-; GFX9-NEXT: v_or_b32_e32 v57, 0x400000, v47
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v47, v47
+; GFX9-NEXT: v_or_b32_e32 v47, 0x400000, v47
; GFX9-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; GFX9-NEXT: v_cndmask_b32_e32 v47, v56, v57, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v47, v56, v47, vcc
; GFX9-NEXT: v_bfe_u32 v56, v7, 16, 1
; GFX9-NEXT: v_add3_u32 v56, v56, v7, s6
-; GFX9-NEXT: v_or_b32_e32 v57, 0x400000, v7
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
-; GFX9-NEXT: v_cndmask_b32_e32 v7, v56, v57, vcc
+; GFX9-NEXT: v_or_b32_e32 v7, 0x400000, v7
+; GFX9-NEXT: v_cndmask_b32_e32 v7, v56, v7, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v56, 16, v8
; GFX9-NEXT: v_add_f32_e32 v56, 0x40c00000, v56
; GFX9-NEXT: v_bfe_u32 v57, v56, 16, 1
; GFX9-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
; GFX9-NEXT: v_add3_u32 v57, v57, v56, s6
-; GFX9-NEXT: v_or_b32_e32 v58, 0x400000, v56
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v56, v56
+; GFX9-NEXT: v_or_b32_e32 v56, 0x400000, v56
; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v56, v57, v58, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v56, v57, v56, vcc
; GFX9-NEXT: v_bfe_u32 v57, v8, 16, 1
; GFX9-NEXT: v_add3_u32 v57, v57, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v58, 0x400000, v8
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v8, v57, v58, vcc
+; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v8
+; GFX9-NEXT: v_cndmask_b32_e32 v8, v57, v8, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v57, 16, v9
; GFX9-NEXT: v_add_f32_e32 v57, 0x40c00000, v57
; GFX9-NEXT: v_bfe_u32 v58, v57, 16, 1
; GFX9-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
; GFX9-NEXT: v_add3_u32 v58, v58, v57, s6
-; GFX9-NEXT: v_or_b32_e32 v59, 0x400000, v57
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v57, v57
+; GFX9-NEXT: v_or_b32_e32 v57, 0x400000, v57
; GFX9-NEXT: v_add_f32_e32 v9, 0x40c00000, v9
-; GFX9-NEXT: v_cndmask_b32_e32 v57, v58, v59, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v57, v58, v57, vcc
; GFX9-NEXT: v_bfe_u32 v58, v9, 16, 1
; GFX9-NEXT: v_add3_u32 v58, v58, v9, s6
-; GFX9-NEXT: v_or_b32_e32 v59, 0x400000, v9
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v9, v9
-; GFX9-NEXT: v_cndmask_b32_e32 v9, v58, v59, vcc
+; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v9
+; GFX9-NEXT: v_cndmask_b32_e32 v9, v58, v9, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v58, 16, v10
; GFX9-NEXT: v_add_f32_e32 v58, 0x40c00000, v58
; GFX9-NEXT: v_bfe_u32 v59, v58, 16, 1
; GFX9-NEXT: v_and_b32_e32 v10, 0xffff0000, v10
; GFX9-NEXT: v_add3_u32 v59, v59, v58, s6
-; GFX9-NEXT: v_or_b32_e32 v60, 0x400000, v58
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v58, v58
+; GFX9-NEXT: v_or_b32_e32 v58, 0x400000, v58
; GFX9-NEXT: v_add_f32_e32 v10, 0x40c00000, v10
-; GFX9-NEXT: v_cndmask_b32_e32 v58, v59, v60, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v58, v59, v58, vcc
; GFX9-NEXT: v_bfe_u32 v59, v10, 16, 1
; GFX9-NEXT: v_add3_u32 v59, v59, v10, s6
-; GFX9-NEXT: v_or_b32_e32 v60, 0x400000, v10
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v10, v10
-; GFX9-NEXT: v_cndmask_b32_e32 v10, v59, v60, vcc
+; GFX9-NEXT: v_or_b32_e32 v10, 0x400000, v10
+; GFX9-NEXT: v_cndmask_b32_e32 v10, v59, v10, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v59, 16, v11
; GFX9-NEXT: v_add_f32_e32 v59, 0x40c00000, v59
; GFX9-NEXT: v_bfe_u32 v60, v59, 16, 1
; GFX9-NEXT: v_and_b32_e32 v11, 0xffff0000, v11
; GFX9-NEXT: v_add3_u32 v60, v60, v59, s6
-; GFX9-NEXT: v_or_b32_e32 v61, 0x400000, v59
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v59, v59
+; GFX9-NEXT: v_or_b32_e32 v59, 0x400000, v59
; GFX9-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
-; GFX9-NEXT: v_cndmask_b32_e32 v59, v60, v61, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v59, v60, v59, vcc
; GFX9-NEXT: v_bfe_u32 v60, v11, 16, 1
; GFX9-NEXT: v_add3_u32 v60, v60, v11, s6
-; GFX9-NEXT: v_or_b32_e32 v61, 0x400000, v11
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v11, v11
-; GFX9-NEXT: v_cndmask_b32_e32 v11, v60, v61, vcc
+; GFX9-NEXT: v_or_b32_e32 v11, 0x400000, v11
+; GFX9-NEXT: v_cndmask_b32_e32 v11, v60, v11, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v60, 16, v12
; GFX9-NEXT: v_add_f32_e32 v60, 0x40c00000, v60
; GFX9-NEXT: v_bfe_u32 v61, v60, 16, 1
; GFX9-NEXT: v_and_b32_e32 v12, 0xffff0000, v12
; GFX9-NEXT: v_add3_u32 v61, v61, v60, s6
-; GFX9-NEXT: v_or_b32_e32 v62, 0x400000, v60
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v60, v60
+; GFX9-NEXT: v_or_b32_e32 v60, 0x400000, v60
; GFX9-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
-; GFX9-NEXT: v_cndmask_b32_e32 v60, v61, v62, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v60, v61, v60, vcc
; GFX9-NEXT: v_bfe_u32 v61, v12, 16, 1
; GFX9-NEXT: v_add3_u32 v61, v61, v12, s6
-; GFX9-NEXT: v_or_b32_e32 v62, 0x400000, v12
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
-; GFX9-NEXT: v_cndmask_b32_e32 v12, v61, v62, vcc
+; GFX9-NEXT: v_or_b32_e32 v12, 0x400000, v12
+; GFX9-NEXT: v_cndmask_b32_e32 v12, v61, v12, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v61, 16, v13
; GFX9-NEXT: v_add_f32_e32 v61, 0x40c00000, v61
; GFX9-NEXT: v_bfe_u32 v62, v61, 16, 1
; GFX9-NEXT: v_and_b32_e32 v13, 0xffff0000, v13
; GFX9-NEXT: v_add3_u32 v62, v62, v61, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v61
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v61, v61
+; GFX9-NEXT: v_or_b32_e32 v61, 0x400000, v61
; GFX9-NEXT: v_add_f32_e32 v13, 0x40c00000, v13
-; GFX9-NEXT: v_cndmask_b32_e32 v61, v62, v63, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v61, v62, v61, vcc
; GFX9-NEXT: v_bfe_u32 v62, v13, 16, 1
; GFX9-NEXT: v_add3_u32 v62, v62, v13, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v13
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v13, v13
-; GFX9-NEXT: v_cndmask_b32_e32 v13, v62, v63, vcc
+; GFX9-NEXT: v_or_b32_e32 v13, 0x400000, v13
+; GFX9-NEXT: v_cndmask_b32_e32 v13, v62, v13, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v62, 16, v14
; GFX9-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
-; GFX9-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; GFX9-NEXT: v_bfe_u32 v63, v62, 16, 1
-; GFX9-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; GFX9-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; GFX9-NEXT: v_add3_u32 v63, v63, v62, s6
-; GFX9-NEXT: v_or_b32_e32 v0, 0x400000, v62
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; GFX9-NEXT: v_bfe_u32 v62, v14, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v1, v63, v0, vcc
-; GFX9-NEXT: v_add3_u32 v62, v62, v14, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v14
+; GFX9-NEXT: v_or_b32_e32 v62, 0x400000, v62
+; GFX9-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; GFX9-NEXT: v_cndmask_b32_e32 v62, v63, v62, vcc
+; GFX9-NEXT: v_bfe_u32 v63, v14, 16, 1
+; GFX9-NEXT: v_add3_u32 v63, v63, v14, s6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v14, v14
-; GFX9-NEXT: v_cndmask_b32_e32 v14, v62, v63, vcc
-; GFX9-NEXT: v_lshlrev_b32_e32 v62, 16, v15
-; GFX9-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
+; GFX9-NEXT: v_or_b32_e32 v14, 0x400000, v14
+; GFX9-NEXT: v_cndmask_b32_e32 v14, v63, v14, vcc
+; GFX9-NEXT: v_lshlrev_b32_e32 v63, 16, v15
+; GFX9-NEXT: v_add_f32_e32 v63, 0x40c00000, v63
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; GFX9-NEXT: v_bfe_u32 v32, v63, 16, 1
; GFX9-NEXT: v_and_b32_e32 v15, 0xffff0000, v15
-; GFX9-NEXT: v_bfe_u32 v63, v62, 16, 1
+; GFX9-NEXT: v_add3_u32 v32, v32, v63, s6
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v63, v63
+; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v63
; GFX9-NEXT: v_add_f32_e32 v15, 0x40c00000, v15
-; GFX9-NEXT: v_add3_u32 v63, v63, v62, s6
-; GFX9-NEXT: v_or_b32_e32 v0, 0x400000, v62
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; GFX9-NEXT: v_bfe_u32 v62, v15, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v0, v63, v0, vcc
-; GFX9-NEXT: v_add3_u32 v62, v62, v15, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v15
+; GFX9-NEXT: v_cndmask_b32_e32 v32, v32, v63, vcc
+; GFX9-NEXT: v_bfe_u32 v63, v15, 16, 1
+; GFX9-NEXT: v_add3_u32 v63, v63, v15, s6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v15, v15
-; GFX9-NEXT: v_cndmask_b32_e32 v15, v62, v63, vcc
+; GFX9-NEXT: v_or_b32_e32 v15, 0x400000, v15
+; GFX9-NEXT: v_cndmask_b32_e32 v15, v63, v15, vcc
; GFX9-NEXT: s_mov_b32 s6, 0x7060302
-; GFX9-NEXT: v_perm_b32 v15, v15, v0, s6
-; GFX9-NEXT: v_perm_b32 v14, v14, v1, s6
-; GFX9-NEXT: v_perm_b32 v1, v16, v41, s6
-; GFX9-NEXT: v_perm_b32 v0, v17, v40, s6
-; GFX9-NEXT: v_perm_b32 v17, v32, v33, s6
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v15, v15, v32, s6
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v14, v14, v62, s6
; GFX9-NEXT: v_perm_b32 v13, v13, v61, s6
; GFX9-NEXT: v_perm_b32 v12, v12, v60, s6
; GFX9-NEXT: v_perm_b32 v11, v11, v59, s6
@@ -210405,6 +210556,8 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v4, v4, v44, s6
; GFX9-NEXT: v_perm_b32 v3, v3, v43, s6
; GFX9-NEXT: v_perm_b32 v2, v2, v42, s6
+; GFX9-NEXT: v_perm_b32 v1, v1, v41, s6
+; GFX9-NEXT: v_perm_b32 v0, v0, v40, s6
; GFX9-NEXT: v_perm_b32 v31, v31, v55, s6
; GFX9-NEXT: v_perm_b32 v30, v30, v54, s6
; GFX9-NEXT: v_perm_b32 v29, v29, v53, s6
@@ -210419,8 +210572,9 @@ define <64 x half> @bitcast_v64bf16_to_v64f16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v20, v20, v36, s6
; GFX9-NEXT: v_perm_b32 v19, v19, v35, s6
; GFX9-NEXT: v_perm_b32 v18, v18, v34, s6
+; GFX9-NEXT: v_perm_b32 v17, v17, v33, s6
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v32, v16, s6
+; GFX9-NEXT: v_perm_b32 v16, v16, v32, s6
; GFX9-NEXT: .LBB100_2: ; %end
; GFX9-NEXT: s_or_b64 exec, exec, s[4:5]
; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
@@ -211589,7 +211743,7 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_or_saveexec_b64 s[4:5], -1
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:716 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
@@ -211611,29 +211765,63 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: v_writelane_b32 v63, s36, 2
; SI-NEXT: v_writelane_b32 v63, s30, 3
; SI-NEXT: v_writelane_b32 v63, s31, 4
-; SI-NEXT: s_and_b32 s6, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s28, 16
-; SI-NEXT: s_and_b32 s28, s19, 0xffff0000
-; SI-NEXT: s_and_b32 s12, s25, 0xffff0000
-; SI-NEXT: v_readfirstlane_b32 s60, v12
+; SI-NEXT: s_and_b32 s41, s16, 0xffff0000
+; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s5, s29, 16
+; SI-NEXT: s_and_b32 s29, s18, 0xffff0000
; SI-NEXT: v_readfirstlane_b32 s34, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s28
-; SI-NEXT: s_and_b32 s61, s60, 0xffff0000
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s41
+; SI-NEXT: s_and_b32 s8, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s27, 16
+; SI-NEXT: s_and_b32 s27, s20, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s29
+; SI-NEXT: s_and_b32 s12, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s25, 16
+; SI-NEXT: s_and_b32 s25, s22, 0xffff0000
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s27
+; SI-NEXT: s_and_b32 s6, s28, 0xffff0000
+; SI-NEXT: v_readfirstlane_b32 s58, v13
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s25
+; SI-NEXT: v_readfirstlane_b32 s56, v14
+; SI-NEXT: s_and_b32 s59, s58, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s6
+; SI-NEXT: v_readfirstlane_b32 s46, v15
+; SI-NEXT: s_and_b32 s57, s56, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s59
+; SI-NEXT: v_readfirstlane_b32 s44, v16
+; SI-NEXT: s_and_b32 s47, s46, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s57
+; SI-NEXT: s_and_b32 s45, s44, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s47
; SI-NEXT: s_and_b32 s40, s17, 0xffff0000
; SI-NEXT: s_lshl_b32 s17, s17, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: s_and_b32 s35, s34, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s45
; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s40
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s35
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s17
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: s_lshl_b32 s7, s28, 16
+; SI-NEXT: s_and_b32 s28, s19, 0xffff0000
; SI-NEXT: s_lshl_b32 s19, s19, 16
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s19
@@ -211645,11 +211833,6 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: s_lshl_b32 s15, s24, 16
; SI-NEXT: s_and_b32 s24, s23, 0xffff0000
; SI-NEXT: s_lshl_b32 s23, s23, 16
-; SI-NEXT: s_lshl_b32 s13, s25, 16
-; SI-NEXT: s_lshl_b32 s9, s27, 16
-; SI-NEXT: s_lshl_b32 s5, s29, 16
-; SI-NEXT: s_and_b32 s31, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s30, s30, 16
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: s_and_b32 s95, s94, 0xffff0000
; SI-NEXT: s_lshl_b32 s94, s94, 16
@@ -211665,120 +211848,97 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: v_readfirstlane_b32 s62, v11
; SI-NEXT: s_and_b32 s63, s62, 0xffff0000
; SI-NEXT: s_lshl_b32 s62, s62, 16
-; SI-NEXT: v_readfirstlane_b32 s58, v13
-; SI-NEXT: s_and_b32 s59, s58, 0xffff0000
; SI-NEXT: s_lshl_b32 s58, s58, 16
-; SI-NEXT: v_readfirstlane_b32 s46, v15
-; SI-NEXT: s_and_b32 s47, s46, 0xffff0000
; SI-NEXT: s_lshl_b32 s46, s46, 16
; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: v_readfirstlane_b32 s44, v16
-; SI-NEXT: v_readfirstlane_b32 s56, v14
+; SI-NEXT: s_and_b32 s43, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_lshl_b32 s18, s18, 16
+; SI-NEXT: v_readfirstlane_b32 s60, v12
; SI-NEXT: v_readfirstlane_b32 s72, v10
; SI-NEXT: v_readfirstlane_b32 s76, v8
; SI-NEXT: v_readfirstlane_b32 s88, v6
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
-; SI-NEXT: s_and_b32 s8, s27, 0xffff0000
-; SI-NEXT: s_and_b32 s25, s22, 0xffff0000
; SI-NEXT: s_lshl_b32 s22, s22, 16
-; SI-NEXT: s_and_b32 s27, s20, 0xffff0000
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_and_b32 s29, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s41, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_and_b32 s43, s42, 0xffff0000
-; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_and_b32 s45, s44, 0xffff0000
; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_and_b32 s57, s56, 0xffff0000
; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_and_b32 s61, s60, 0xffff0000
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s21
; SI-NEXT: s_lshl_b32 s60, s60, 16
; SI-NEXT: s_and_b32 s73, s72, 0xffff0000
; SI-NEXT: s_lshl_b32 s72, s72, 16
; SI-NEXT: s_and_b32 s77, s76, 0xffff0000
; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s21
; SI-NEXT: s_and_b32 s89, s88, 0xffff0000
; SI-NEXT: s_lshl_b32 s88, s88, 16
; SI-NEXT: s_and_b32 s93, s92, 0xffff0000
; SI-NEXT: s_lshl_b32 s92, s92, 16
; SI-NEXT: s_and_b32 vcc_hi, vcc_lo, 0xffff0000
; SI-NEXT: s_lshl_b32 vcc_lo, vcc_lo, 16
-; SI-NEXT: s_and_b32 s35, s34, 0xffff0000
+; SI-NEXT: s_and_b32 s31, s30, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s30, 16
; SI-NEXT: s_lshl_b32 s34, s34, 16
; SI-NEXT: v_readfirstlane_b32 s36, v18
; SI-NEXT: s_cmp_lg_u32 s36, 0
-; SI-NEXT: v_mul_f32_e64 v48, 1.0, s41
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s29
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s27
-; SI-NEXT: v_mul_f32_e64 v36, 1.0, s26
-; SI-NEXT: v_mul_f32_e64 v62, 1.0, s25
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s24
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s14
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s10
-; SI-NEXT: v_mul_f32_e64 v38, 1.0, s8
-; SI-NEXT: v_mul_f32_e64 v60, 1.0, s6
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s4
-; SI-NEXT: v_mul_f32_e64 v58, 1.0, s35
-; SI-NEXT: v_mul_f32_e64 v34, 1.0, s31
-; SI-NEXT: v_mul_f32_e64 v56, 1.0, vcc_hi
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s28
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s26
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s24
+; SI-NEXT: v_mul_f32_e64 v44, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v62, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s4
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s31
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, vcc_hi
; SI-NEXT: v_mul_f32_e64 v29, 1.0, s95
; SI-NEXT: v_mul_f32_e64 v4, 1.0, s93
-; SI-NEXT: v_mul_f32_e64 v33, 1.0, s91
-; SI-NEXT: v_mul_f32_e64 v42, 1.0, s89
-; SI-NEXT: v_mul_f32_e64 v54, 1.0, s79
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s77
-; SI-NEXT: v_mul_f32_e64 v45, 1.0, s75
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s73
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s63
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s59
-; SI-NEXT: v_mul_f32_e64 v47, 1.0, s57
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s47
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s45
-; SI-NEXT: v_mul_f32_e64 v32, 1.0, s43
-; SI-NEXT: v_mul_f32_e64 v44, 1.0, s42
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s18
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s20
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s22
-; SI-NEXT: v_mul_f32_e64 v35, 1.0, s15
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s11
-; SI-NEXT: v_mul_f32_e64 v59, 1.0, s7
-; SI-NEXT: v_mul_f32_e64 v57, 1.0, s34
-; SI-NEXT: v_mul_f32_e64 v55, 1.0, vcc_lo
-; SI-NEXT: v_mul_f32_e64 v49, 1.0, s92
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s88
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s76
-; SI-NEXT: v_mul_f32_e64 v37, 1.0, s72
-; SI-NEXT: v_mul_f32_e64 v61, 1.0, s60
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s56
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s44
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v32, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s77
+; SI-NEXT: v_mul_f32_e64 v33, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v36, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v42, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s30
+; SI-NEXT: v_mul_f32_e64 v60, 1.0, s20
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s22
+; SI-NEXT: v_mul_f32_e64 v48, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v52, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s7
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s23
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v59, 1.0, s34
+; SI-NEXT: v_mul_f32_e64 v54, 1.0, vcc_lo
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v50, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v45, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v43, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v41, 1.0, s44
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s13
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s9
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s5
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s30
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
@@ -211787,713 +211947,663 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s90
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s78
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s74
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s62
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s58
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s46
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s46
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s18
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:592 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:596 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:600 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:604 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:608 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:612 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:616 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:620 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:624 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:628 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
; SI-NEXT: s_cbranch_scc0 .LBB101_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_mov_b32_e32 v46, v22
-; SI-NEXT: v_mov_b32_e32 v22, v13
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v1
-; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v5, v14
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v41, v16
-; SI-NEXT: v_mov_b32_e32 v43, v49
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v3
-; SI-NEXT: v_mov_b32_e32 v39, v17
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
-; SI-NEXT: v_and_b32_e32 v50, 0xffff0000, v48
-; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v4
-; SI-NEXT: v_mov_b32_e32 v24, v36
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:688 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: v_and_b32_e32 v39, 0xffff0000, v62
-; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:636 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v9
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v8, v26
+; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v24
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v1
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v31
+; SI-NEXT: v_mov_b32_e32 v11, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v36
+; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v62
; SI-NEXT: s_mov_b64 s[4:5], 0
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v9
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(4)
+; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:672 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v9
-; SI-NEXT: v_mov_b32_e32 v49, v9
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshr_b64 v[51:52], v[0:1], 16
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v52, v51
-; SI-NEXT: v_mov_b32_e32 v51, v50
-; SI-NEXT: v_mov_b32_e32 v50, v4
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v10
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v46
+; SI-NEXT: v_mov_b32_e32 v45, v9
+; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v29
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_and_b32_e32 v56, 0xffff0000, v10
+; SI-NEXT: v_and_b32_e32 v39, 0xffff0000, v37
+; SI-NEXT: v_mov_b32_e32 v24, v37
+; SI-NEXT: v_lshrrev_b32_e32 v61, 16, v24
+; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v46
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshr_b64 v[13:14], v[0:1], 16
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[57:58], v[6:7], 16
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v36
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:648 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:652 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:656 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v21
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_lshr_b64 v[13:14], v[0:1], 16
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:660 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:664 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:668 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v25
+; SI-NEXT: v_and_b32_e32 v25, 0xffff0000, v30
+; SI-NEXT: v_mov_b32_e32 v58, v59
+; SI-NEXT: v_lshrrev_b32_e32 v59, 16, v11
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v13, v52
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v42
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v28
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v28
+; SI-NEXT: v_lshr_b64 v[40:41], v[6:7], 16
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v13
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v4
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v10
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_mov_b32_e32 v15, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
-; SI-NEXT: v_and_b32_e32 v10, 0xffff0000, v56
-; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v56
-; SI-NEXT: v_lshr_b64 v[40:41], v[15:16], 16
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshr_b64 v[8:9], v[0:1], 16
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v38
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:692 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:696 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:700 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v2
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:632 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v15
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v44
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v27
-; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v25
-; SI-NEXT: v_lshr_b64 v[8:9], v[0:1], 16
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v20
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v34
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v60
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v28
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v60
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: v_lshr_b64 v[8:9], v[0:1], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v8, v2
-; SI-NEXT: v_lshr_b64 v[1:2], v[19:20], 16
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v58
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v29
-; SI-NEXT: v_mov_b32_e32 v9, v3
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v45
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v32
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v58
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
-; SI-NEXT: v_lshr_b64 v[11:12], v[0:1], 16
+; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:676 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:680 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:684 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v33
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:704 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:708 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:712 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v10, 0xffff0000, v26
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: v_lshr_b64 v[19:20], v[0:1], 16
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v53, 0xffff0000, v17
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[48:49], v[6:7], 16
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v54
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: v_lshr_b64 v[15:16], v[0:1], 16
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v19
+; SI-NEXT: v_lshr_b64 v[19:20], v[34:35], 16
+; SI-NEXT: v_and_b32_e32 v20, 0xffff0000, v2
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[54:55], v[7:8], 16
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v16, v35
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v30
-; SI-NEXT: v_mov_b32_e32 v2, v0
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v0, v26
+; SI-NEXT: v_lshr_b64 v[13:14], v[7:8], 16
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_mov_b32_e32 v33, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_mov_b32_e32 v37, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v31
-; SI-NEXT: v_lshr_b64 v[11:12], v[33:34], 16
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v12, v51
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v6
-; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: v_lshr_b64 v[32:33], v[44:45], 16
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v50
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v34, 0xffff0000, v47
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
-; SI-NEXT: v_lshr_b64 v[28:29], v[37:38], 16
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v32
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_mov_b32_e32 v8, v7
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[21:22], v[14:15], 16
; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v49
-; SI-NEXT: v_mov_b32_e32 v49, v40
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_mov_b32_e32 v23, v52
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
-; SI-NEXT: v_lshr_b64 v[35:36], v[3:4], 16
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v33
+; SI-NEXT: v_lshr_b64 v[26:27], v[8:9], 16
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v3, v46
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v48
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:640 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:644 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v8, v4
+; SI-NEXT: v_mov_b32_e32 v22, v2
+; SI-NEXT: v_mov_b32_e32 v27, v50
+; SI-NEXT: v_and_b32_e32 v50, 0xffff0000, v42
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[4:5], v[14:15], 16
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v5, v30
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mov_b32_e32 v30, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mov_b32_e32 v32, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mov_b32_e32 v34, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v21
-; SI-NEXT: v_mov_b32_e32 v36, v22
-; SI-NEXT: v_mov_b32_e32 v48, v39
-; SI-NEXT: v_mov_b32_e32 v22, v51
-; SI-NEXT: v_mov_b32_e32 v46, v43
-; SI-NEXT: v_lshr_b64 v[53:54], v[22:23], 16
-; SI-NEXT: v_mov_b32_e32 v54, v15
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v15
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v7
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v15
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[30:31], 16
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v2, v36
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v62
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v45
+; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v8
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v2
+; SI-NEXT: v_mov_b32_e32 v2, v26
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v36, 0xffff0000, v6
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v2, v21
+; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v7
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[51:52], v[32:33], 16
+; SI-NEXT: v_lshr_b64 v[32:33], v[34:35], 16
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v10
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v42
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v5
+; SI-NEXT: v_mov_b32_e32 v35, v19
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[37:38], v[14:15], 16
+; SI-NEXT: s_waitcnt expcnt(3)
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(2)
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v30
-; SI-NEXT: v_lshr_b64 v[29:30], v[12:13], 16
-; SI-NEXT: v_mov_b32_e32 v30, v35
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v8
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v44
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v0
-; SI-NEXT: v_mov_b32_e32 v0, v25
-; SI-NEXT: v_lshrrev_b32_e32 v62, 16, v0
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v22
+; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v16
+; SI-NEXT: v_lshr_b64 v[2:3], v[3:4], 16
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v47
-; SI-NEXT: v_mov_b32_e32 v47, v44
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v2, v1
+; SI-NEXT: v_mov_b32_e32 v38, v40
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v42
+; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v6
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v6
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:692 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:696 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:700 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:664 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:668 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_mov_b32_e32 v52, v23
-; SI-NEXT: v_mov_b32_e32 v45, v4
-; SI-NEXT: v_mov_b32_e32 v44, v3
-; SI-NEXT: v_lshr_b64 v[3:4], v[50:51], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v62
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v11, v48
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v3, v49
-; SI-NEXT: v_mov_b32_e32 v43, v45
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[3:4], v[48:49], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[3:4], v[44:45], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v17
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v5, v7
+; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_mov_b32_e32 v50, v19
-; SI-NEXT: v_lshr_b64 v[0:1], v[1:2], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[6:7], v[56:57], 16
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v11
-; SI-NEXT: v_lshr_b64 v[48:49], v[48:49], 16
-; SI-NEXT: v_mov_b32_e32 v49, v46
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_mov_b32_e32 v44, v4
-; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[6:7], v[39:40], 16
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v14, v57
+; SI-NEXT: v_lshr_b64 v[39:40], v[23:24], 16
+; SI-NEXT: s_waitcnt vmcnt(1) expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v7, v15
+; SI-NEXT: v_mov_b32_e32 v8, v16
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:676 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:680 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:684 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_mov_b32_e32 v57, v16
+; SI-NEXT: v_mov_b32_e32 v56, v15
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:704 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[7:8], v[56:57], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:708 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:712 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_mov_b32_e32 v12, v4
-; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[7:8], v[47:48], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[3:4], v[18:19], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[7:8], v[53:54], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[3:4], v[14:15], 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v3, v2
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshr_b64 v[0:1], v[10:11], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[7:8], v[12:13], 16
+; SI-NEXT: v_lshr_b64 v[12:13], v[0:1], 16
+; SI-NEXT: v_mov_b32_e32 v1, v51
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v15, v4
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[7:8], v[25:26], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v47, v54
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v28
-; SI-NEXT: v_mov_b32_e32 v15, v16
-; SI-NEXT: v_mov_b32_e32 v16, v17
-; SI-NEXT: v_mov_b32_e32 v19, v36
-; SI-NEXT: v_mov_b32_e32 v17, v39
-; SI-NEXT: v_mov_b32_e32 v18, v51
-; SI-NEXT: v_mov_b32_e32 v14, v6
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshr_b64 v[0:1], v[27:28], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[7:8], v[18:19], 16
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[34:35], 16
+; SI-NEXT: v_lshr_b64 v[7:8], v[20:21], 16
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[7:8], v[50:51], 16
+; SI-NEXT: v_mov_b32_e32 v51, v28
+; SI-NEXT: v_mov_b32_e32 v50, v27
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:660 ; 4-byte Folded Reload
; SI-NEXT: v_mov_b32_e32 v1, v32
-; SI-NEXT: v_lshr_b64 v[32:33], v[31:32], 16
-; SI-NEXT: v_lshr_b64 v[33:34], v[6:7], 16
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:672 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v35, v15
-; SI-NEXT: v_mov_b32_e32 v28, v48
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v31, v0
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshr_b64 v[16:17], v[36:37], 16
+; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
+; SI-NEXT: v_mov_b32_e32 v56, v37
+; SI-NEXT: v_lshr_b64 v[36:37], v[33:34], 16
+; SI-NEXT: v_mov_b32_e32 v34, v57
+; SI-NEXT: v_mov_b32_e32 v32, v16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_mov_b32_e32 v22, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[0:1], v[29:30], 16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[36:37], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[1:2], v[60:61], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v37, v8
-; SI-NEXT: v_mov_b32_e32 v36, v24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[48:49], 16
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[15:16], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[52:53], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[25:26], 16
-; SI-NEXT: v_mov_b32_e32 v26, v29
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[3:4], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[59:60], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[58:59], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(14) expcnt(0)
+; SI-NEXT: v_lshr_b64 v[1:2], v[54:55], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[57:58], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[18:19], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[55:56], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[26:27], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[46:47], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[50:51], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[41:42], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[45:46], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[39:40], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[9:10], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[8:9], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[43:44], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[20:21], 16
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_mov_b32_e32 v16, v41
-; SI-NEXT: v_lshr_b64 v[39:40], v[61:62], 16
-; SI-NEXT: v_mov_b32_e32 v8, v20
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v11, v10
-; SI-NEXT: v_mov_b32_e32 v10, v13
+; SI-NEXT: v_lshr_b64 v[1:2], v[41:42], 16
+; SI-NEXT: v_mov_b32_e32 v30, v7
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v7, v29
+; SI-NEXT: v_mov_b32_e32 v29, v12
; SI-NEXT: s_branch .LBB101_3
; SI-NEXT: .LBB101_2:
-; SI-NEXT: v_mov_b32_e32 v19, v13
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v9, v17
+; SI-NEXT: s_waitcnt expcnt(6)
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr26
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr53
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: v_mov_b32_e32 v58, v59
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr30
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr32
-; SI-NEXT: ; implicit-def: $vgpr10
-; SI-NEXT: ; implicit-def: $vgpr52
-; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr43
-; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr5
+; SI-NEXT: ; implicit-def: $vgpr14
+; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr11
+; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr12
-; SI-NEXT: ; implicit-def: $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr54
-; SI-NEXT: ; implicit-def: $vgpr3
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr30
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr15
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_mov_b32_e32 v33, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mov_b32_e32 v7, v16
; SI-NEXT: .LBB101_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -212501,528 +212611,544 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: s_cbranch_scc1 .LBB101_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v33, 0xffff0000, v8
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v13, v19
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v13
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v60
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v11
+; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v48
+; SI-NEXT: v_and_b32_e32 v48, 0xffff0000, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v35, 0xffff0000, v58
+; SI-NEXT: v_and_b32_e32 v30, 0xffff0000, v9
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v59
-; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v61
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v57
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v58, 0xffff0000, v15
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v6
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v36, 0xffff0000, v18
+; SI-NEXT: v_and_b32_e32 v37, 0xffff0000, v50
+; SI-NEXT: v_and_b32_e32 v46, 0xffff0000, v45
+; SI-NEXT: v_and_b32_e32 v45, 0xffff0000, v43
+; SI-NEXT: v_and_b32_e32 v49, 0xffff0000, v23
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v33
+; SI-NEXT: v_and_b32_e32 v53, 0xffff0000, v52
+; SI-NEXT: v_and_b32_e32 v52, 0xffff0000, v54
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
+; SI-NEXT: v_and_b32_e32 v56, 0xffff0000, v26
+; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v41
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
+; SI-NEXT: v_add_f32_e32 v57, 0x40c00000, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v15
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v11
+; SI-NEXT: v_add_f32_e32 v11, 0x40c00000, v18
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v31, 0x40c00000, v15
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v2
+; SI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v4
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:656 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
+; SI-NEXT: v_add_f32_e32 v9, 0x40c00000, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v22
+; SI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v4
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v17
+; SI-NEXT: v_add_f32_e32 v50, 0x40c00000, v18
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v20
+; SI-NEXT: v_add_f32_e32 v59, 0x40c00000, v18
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
+; SI-NEXT: v_add_f32_e32 v40, 0x40c00000, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v23
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v61, 0x40c00000, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v25
+; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v14
+; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:652 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v17
+; SI-NEXT: v_add_f32_e32 v42, 0x40c00000, v18
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v23
+; SI-NEXT: v_add_f32_e32 v33, 0x40c00000, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
+; SI-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v16, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:648 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v16, 0x40c00000, v16
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v16
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v17
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:644 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v19, 0x40c00000, v19
+; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v19
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v16
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(12)
-; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v27
-; SI-NEXT: v_add_f32_e32 v4, 0x40c00000, v3
-; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v29
-; SI-NEXT: v_add_f32_e32 v18, 0x40c00000, v18
-; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v18
-; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v34
+; SI-NEXT: v_add_f32_e32 v12, 0x40c00000, v17
+; SI-NEXT: v_lshr_b64 v[16:17], v[31:32], 16
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:640 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v19, 0x40c00000, v19
+; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v19
+; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:636 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v19, 0x40c00000, v19
+; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v19
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:632 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v19, 0x40c00000, v19
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v19
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v18, 0x40c00000, v18
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:628 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v10, 0xffff0000, v22
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v14
-; SI-NEXT: v_and_b32_e32 v48, 0xffff0000, v17
-; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v38
-; SI-NEXT: v_and_b32_e32 v11, 0xffff0000, v36
-; SI-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v11
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v10
-; SI-NEXT: v_and_b32_e32 v32, 0xffff0000, v55
-; SI-NEXT: v_and_b32_e32 v28, 0xffff0000, v35
-; SI-NEXT: v_and_b32_e32 v35, 0xffff0000, v25
-; SI-NEXT: v_and_b32_e32 v30, 0xffff0000, v49
-; SI-NEXT: v_and_b32_e32 v37, 0xffff0000, v37
-; SI-NEXT: v_lshrrev_b32_e32 v62, 16, v4
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v61, 0x40c00000, v2
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v5
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v24, 0x40c00000, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v8
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:616 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v3, 0x40c00000, v2
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v12
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v0
-; SI-NEXT: v_add_f32_e32 v59, 0x40c00000, v0
+; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v18
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v5
-; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v5
-; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:624 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v20, 0x40c00000, v19
+; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v20
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: v_add_f32_e32 v9, 0x40c00000, v8
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v50, 0x40c00000, v2
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v6
-; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v9
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v40, 0x40c00000, v2
-; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v44, 0x40c00000, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v18
-; SI-NEXT: v_add_f32_e32 v26, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v12
-; SI-NEXT: v_add_f32_e32 v13, 0x40c00000, v12
-; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v13
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v12, 0x40c00000, v2
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v19, 0x40c00000, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v52, 0x40c00000, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v1
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v1
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
-; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v14
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v0
-; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
+; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v2
+; SI-NEXT: v_add_f32_e32 v21, 0x40c00000, v21
+; SI-NEXT: v_lshrrev_b32_e32 v62, 16, v21
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:620 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v7
-; SI-NEXT: v_add_f32_e32 v38, 0x40c00000, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v54, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v11, 0xffff0000, v21
+; SI-NEXT: v_add_f32_e32 v38, 0x40c00000, v11
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v11, 0xffff0000, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v28
+; SI-NEXT: v_add_f32_e32 v28, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v11
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v21
-; SI-NEXT: v_add_f32_e32 v42, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
+; SI-NEXT: v_add_f32_e32 v55, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[38:39], v[38:39], 16
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v34, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v34
-; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_f32_e32 v44, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v56, 0x40c00000, v2
+; SI-NEXT: v_add_f32_e32 v54, 0x40c00000, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v8
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v55
+; SI-NEXT: v_lshr_b64 v[18:19], v[7:8], 16
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v27
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v44
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v56
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
-; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
-; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v54
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v58
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v49
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v57, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v11, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:612 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v57
-; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v11
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v58, 0x40c00000, v2
+; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
-; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v7
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v14
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:608 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v53
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
-; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
-; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v14
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v28
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v14
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v35
+; SI-NEXT: v_add_f32_e32 v49, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v49
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v48
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v28, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v28
+; SI-NEXT: v_add_f32_e32 v53, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:604 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v53
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v35
+; SI-NEXT: v_and_b32_e32 v35, 0xffff0000, v11
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v29, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v29
-; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v18
-; SI-NEXT: v_add_f32_e32 v18, 0x40c00000, v18
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v18
-; SI-NEXT: v_add_f32_e32 v18, 0x40c00000, v17
-; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v18
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: v_add_f32_e32 v25, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:600 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v25
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v52
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v21, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v21
-; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v15
-; SI-NEXT: v_add_f32_e32 v16, 0x40c00000, v15
-; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v16
-; SI-NEXT: v_lshr_b64 v[16:17], v[50:51], 16
-; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v57
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v26, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:596 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v32
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v26
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v36
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v22, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v22
+; SI-NEXT: v_add_f32_e32 v27, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:592 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v30
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v27
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v56
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v27
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v23, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v23
+; SI-NEXT: v_add_f32_e32 v52, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
-; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v23
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v52
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v37
+; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v52
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v30, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v30
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v1
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v36, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v48
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v36
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v46
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v35, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v35
+; SI-NEXT: v_add_f32_e32 v37, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v37
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v37
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v30
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v36, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v36
+; SI-NEXT: v_add_f32_e32 v48, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v47
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v48
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v45
+; SI-NEXT: v_and_b32_e32 v56, 0xffff0000, v48
+; SI-NEXT: v_and_b32_e32 v48, 0xffff0000, v49
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v37, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v37
-; SI-NEXT: v_lshr_b64 v[46:47], v[10:11], 16
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v33
-; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_f32_e32 v30, 0x40c00000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v30
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v47
+; SI-NEXT: v_lshr_b64 v[46:47], v[57:58], 16
+; SI-NEXT: v_and_b32_e32 v45, 0xffff0000, v30
+; SI-NEXT: v_and_b32_e32 v30, 0xffff0000, v26
+; SI-NEXT: v_lshr_b64 v[26:27], v[33:34], 16
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v2
-; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v10, 0xffff0000, v10
-; SI-NEXT: v_add_f32_e32 v48, 0x40c00000, v10
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v31
-; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v2
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v2
+; SI-NEXT: v_and_b32_e32 v22, 0xffff0000, v2
; SI-NEXT: v_mov_b32_e32 v2, v1
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v48
-; SI-NEXT: v_lshr_b64 v[7:8], v[10:11], 16
-; SI-NEXT: v_mov_b32_e32 v10, v60
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshr_b64 v[8:9], v[24:25], 16
-; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v48
-; SI-NEXT: v_lshr_b64 v[48:49], v[59:60], 16
-; SI-NEXT: v_and_b32_e32 v59, 0xffff0000, v37
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v36
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v21
-; SI-NEXT: v_lshr_b64 v[36:37], v[26:27], 16
-; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v14
-; SI-NEXT: v_and_b32_e32 v49, 0xffff0000, v22
-; SI-NEXT: v_lshr_b64 v[22:23], v[19:20], 16
-; SI-NEXT: v_lshr_b64 v[18:19], v[38:39], 16
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v35
-; SI-NEXT: v_and_b32_e32 v24, 0xffff0000, v28
-; SI-NEXT: v_and_b32_e32 v35, 0xffff0000, v29
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[10:11], v[44:45], 16
-; SI-NEXT: v_lshr_b64 v[60:61], v[61:62], 16
-; SI-NEXT: v_mov_b32_e32 v29, v10
-; SI-NEXT: v_mov_b32_e32 v28, v9
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v34
-; SI-NEXT: v_lshr_b64 v[10:11], v[42:43], 16
-; SI-NEXT: v_mov_b32_e32 v39, v46
-; SI-NEXT: v_mov_b32_e32 v43, v22
-; SI-NEXT: v_lshr_b64 v[32:33], v[0:1], 16
-; SI-NEXT: v_mov_b32_e32 v1, v62
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v25
-; SI-NEXT: v_lshr_b64 v[25:26], v[12:13], 16
-; SI-NEXT: v_lshr_b64 v[13:14], v[52:53], 16
-; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v58
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: v_lshr_b64 v[52:53], v[54:55], 16
-; SI-NEXT: v_lshr_b64 v[26:27], v[9:10], 16
-; SI-NEXT: v_mov_b32_e32 v44, v25
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v4
-; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v30
-; SI-NEXT: v_mov_b32_e32 v30, v32
-; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
-; SI-NEXT: v_lshr_b64 v[32:33], v[47:48], 16
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v37
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v36
+; SI-NEXT: v_lshr_b64 v[36:37], v[61:62], 16
+; SI-NEXT: v_and_b32_e32 v37, 0xffff0000, v54
+; SI-NEXT: v_lshr_b64 v[23:24], v[0:1], 16
+; SI-NEXT: v_mov_b32_e32 v1, v58
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v6
-; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v10
+; SI-NEXT: v_lshr_b64 v[57:58], v[9:10], 16
+; SI-NEXT: v_lshr_b64 v[8:9], v[5:6], 16
+; SI-NEXT: v_lshr_b64 v[10:11], v[28:29], 16
+; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v55
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v51
-; SI-NEXT: v_lshr_b64 v[50:51], v[40:41], 16
-; SI-NEXT: v_and_b32_e32 v51, 0xffff0000, v56
-; SI-NEXT: v_lshr_b64 v[53:54], v[51:52], 16
-; SI-NEXT: v_mov_b32_e32 v54, v5
+; SI-NEXT: v_mov_b32_e32 v1, v6
+; SI-NEXT: v_lshr_b64 v[5:6], v[59:60], 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v0, v2
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v3
-; SI-NEXT: v_lshr_b64 v[2:3], v[17:18], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v4
+; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v25
+; SI-NEXT: v_and_b32_e32 v25, 0xffff0000, v44
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[12:13], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v32
+; SI-NEXT: v_lshr_b64 v[31:32], v[50:51], 16
+; SI-NEXT: v_lshr_b64 v[49:50], v[12:13], 16
+; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v14
+; SI-NEXT: v_lshr_b64 v[13:14], v[40:41], 16
+; SI-NEXT: v_mov_b32_e32 v33, v12
+; SI-NEXT: v_mov_b32_e32 v34, v13
+; SI-NEXT: v_lshr_b64 v[11:12], v[25:26], 16
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[21:22], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[11:12], v[37:38], 16
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[24:25], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v2, v36
-; SI-NEXT: v_mov_b32_e32 v12, v50
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[11:12], v[35:36], 16
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[35:36], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[11:12], v[33:34], 16
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[12:13], v[48:49], 16
+; SI-NEXT: v_and_b32_e32 v51, 0xffff0000, v53
+; SI-NEXT: v_lshr_b64 v[52:53], v[42:43], 16
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[12:13], v[51:52], 16
+; SI-NEXT: v_mov_b32_e32 v14, v26
+; SI-NEXT: v_mov_b32_e32 v11, v49
+; SI-NEXT: v_mov_b32_e32 v47, v52
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v2, v29
-; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[49:50], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[15:16], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[2:3], v[4:5], 16
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v50, v16
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v3, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshr_b64 v[20:21], v[0:1], 16
+; SI-NEXT: v_mov_b32_e32 v0, v4
+; SI-NEXT: v_mov_b32_e32 v1, v5
+; SI-NEXT: v_mov_b32_e32 v5, v10
+; SI-NEXT: v_mov_b32_e32 v4, v36
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mov_b32_e32 v4, v1
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mov_b32_e32 v0, v18
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v1, v31
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mov_b32_e32 v1, v16
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[17:18], v[30:31], 16
+; SI-NEXT: v_lshr_b64 v[29:30], v[7:8], 16
+; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
+; SI-NEXT: v_lshr_b64 v[30:31], v[56:57], 16
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[31:32], v[45:46], 16
+; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[17:18], v[19:20], 16
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[32:33], v[22:23], 16
+; SI-NEXT: v_mov_b32_e32 v35, v20
+; SI-NEXT: v_mov_b32_e32 v21, v60
+; SI-NEXT: v_mov_b32_e32 v56, v23
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v15, v3
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[1:2], v[2:3], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mov_b32_e32 v1, v8
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshr_b64 v[0:1], v[7:8], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v60
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v57
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshr_b64 v[0:1], v[59:60], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v48
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v46
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
; SI-NEXT: .LBB101_5: ; %end
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v26
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v10
-; SI-NEXT: v_and_b32_e32 v25, 0xffff, v3
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v52
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v18
-; SI-NEXT: v_and_b32_e32 v19, 0xffff, v12
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v43
-; SI-NEXT: v_and_b32_e32 v11, 0xffff, v44
-; SI-NEXT: v_and_b32_e32 v21, 0xffff, v50
-; SI-NEXT: v_and_b32_e32 v23, 0xffff, v54
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v5
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:340 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v14
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v38
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v15
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:368 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:372 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v10, 0xffff, v34
+; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v47
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v56
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
@@ -213043,142 +213169,134 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: v_readlane_b32 s36, v63, 2
; SI-NEXT: v_readlane_b32 s35, v63, 1
; SI-NEXT: v_readlane_b32 s34, v63, 0
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v14
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff, v8
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v17, 0xffff, v14
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v14
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v8
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v4
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:344 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:348 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v0, v0, v1
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v27
; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v53
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v2, v3
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v8
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v36
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v16, 0xffff, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v19, 0xffff, v35
+; SI-NEXT: v_and_b32_e32 v23, 0xffff, v23
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_or_b32_e32 v2, v2, v4
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v9
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:360 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:364 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v17, 0xffff, v18
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v9
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:352 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:356 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v39
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v6, v6, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_or_b32_e32 v6, v6, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:332 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
+; SI-NEXT: v_or_b32_e32 v7, v7, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v12
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_or_b32_e32 v8, v8, v9
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v13
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:340 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:344 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v15
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v12
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v10, 0xffff, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:376 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:380 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v10, v10, v12
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v13
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v15
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v11, v11, v12
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:348 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:352 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_or_b32_e32 v12, v12, v13
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v12, 0xffff, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:384 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_or_b32_e32 v12, v12, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:356 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:360 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
+; SI-NEXT: v_or_b32_e32 v13, v13, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v28
-; SI-NEXT: v_or_b32_e32 v13, v16, v13
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v26
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_or_b32_e32 v14, v14, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:364 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:368 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v21
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
-; SI-NEXT: v_or_b32_e32 v16, v16, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v16, 0xffff, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v16, v16, v18
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:372 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:376 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v21
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v17, v17, v18
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; SI-NEXT: v_or_b32_e32 v18, v18, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v18, 0xffff, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_or_b32_e32 v18, v18, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:380 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:384 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
+; SI-NEXT: v_or_b32_e32 v19, v19, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v20, 0xffff, v20
; SI-NEXT: s_waitcnt vmcnt(1)
@@ -213189,82 +213307,81 @@ define inreg <64 x half> @bitcast_v64bf16_to_v64f16_scalar(<64 x bfloat> inreg %
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_or_b32_e32 v22, v22, v23
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v29
; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_or_b32_e32 v24, v24, v25
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v27, 0xffff, v26
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
; SI-NEXT: v_or_b32_e32 v25, v27, v25
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v26, 0xffff, v39
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v30
+; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
; SI-NEXT: v_or_b32_e32 v26, v26, v27
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v29, 0xffff, v28
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v28
; SI-NEXT: v_or_b32_e32 v27, v29, v27
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v31
-; SI-NEXT: v_and_b32_e32 v31, 0xffff, v30
; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
; SI-NEXT: v_or_b32_e32 v28, v28, v29
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v31, 0xffff, v30
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v30
; SI-NEXT: v_or_b32_e32 v29, v31, v29
-; SI-NEXT: v_and_b32_e32 v30, 0xffff, v33
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:332 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v32
+; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
; SI-NEXT: v_or_b32_e32 v30, v30, v31
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v31, 0xffff, v32
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v33
-; SI-NEXT: v_or_b32_e32 v31, v31, v32
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v32
+; SI-NEXT: v_or_b32_e32 v31, v33, v31
; SI-NEXT: s_or_saveexec_b64 s[4:5], -1
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:716 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:688 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -217456,8 +217573,8 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_or_saveexec_b64 s[4:5], -1
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:48 ; 4-byte Folded Spill
@@ -217640,20 +217757,20 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: s_lshl_b32 s54, s63, 16
; SI-NEXT: s_lshl_b32 s55, s72, 16
; SI-NEXT: s_lshl_b32 s64, s73, 16
-; SI-NEXT: s_lshl_b32 s6, s74, 16
-; SI-NEXT: s_lshl_b32 s65, s75, 16
-; SI-NEXT: s_lshl_b32 s7, s76, 16
-; SI-NEXT: s_lshl_b32 s66, s77, 16
-; SI-NEXT: s_lshl_b32 s67, s78, 16
-; SI-NEXT: s_lshl_b32 s68, s79, 16
-; SI-NEXT: s_lshl_b32 s8, s88, 16
-; SI-NEXT: s_lshl_b32 s69, s89, 16
+; SI-NEXT: s_lshl_b32 s65, s74, 16
+; SI-NEXT: s_lshl_b32 s66, s75, 16
+; SI-NEXT: s_lshl_b32 s67, s76, 16
+; SI-NEXT: s_lshl_b32 s6, s77, 16
+; SI-NEXT: s_lshl_b32 s68, s78, 16
+; SI-NEXT: s_lshl_b32 s7, s79, 16
+; SI-NEXT: s_lshl_b32 s69, s88, 16
+; SI-NEXT: s_lshl_b32 s8, s89, 16
; SI-NEXT: s_lshl_b32 s70, s90, 16
-; SI-NEXT: s_lshl_b32 s9, s91, 16
-; SI-NEXT: s_lshl_b32 s71, s92, 16
+; SI-NEXT: s_lshl_b32 s71, s91, 16
+; SI-NEXT: s_lshl_b32 s9, s92, 16
; SI-NEXT: s_lshl_b32 s80, s93, 16
-; SI-NEXT: s_lshl_b32 s10, s94, 16
-; SI-NEXT: s_lshl_b32 s81, s95, 16
+; SI-NEXT: s_lshl_b32 s81, s94, 16
+; SI-NEXT: s_lshl_b32 s10, s95, 16
; SI-NEXT: s_lshl_b32 s82, vcc_lo, 16
; SI-NEXT: s_lshl_b32 s11, vcc_hi, 16
; SI-NEXT: s_mov_b32 s34, s4
@@ -217691,18 +217808,18 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: ; implicit-def: $sgpr66
; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr8
; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr71
+; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr11
; SI-NEXT: ; implicit-def: $sgpr83
@@ -217803,14 +217920,14 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v0, vcc_lo
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v1, s95
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v0, s94
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
@@ -217819,79 +217936,78 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v51, v1
+; SI-NEXT: v_cvt_f16_f32_e32 v7, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s91
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v0, s92
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v7, v1
+; SI-NEXT: v_cvt_f16_f32_e32 v8, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s89
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
-; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v39, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s79
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_cvt_f16_f32_e32 v48, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s90
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v16, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s79
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v8, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v49, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s88
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v53, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s75
+; SI-NEXT: v_cvt_f16_f32_e32 v52, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s77
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v49, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v51, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s78
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v55, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s73
+; SI-NEXT: v_cvt_f16_f32_e32 v54, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s75
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v52, v0
-; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v55
-; SI-NEXT: v_lshlrev_b32_e32 v55, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: v_cvt_f16_f32_e32 v53, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s76
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v41, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s63
+; SI-NEXT: v_cvt_f16_f32_e32 v40, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s73
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v54, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v55, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s74
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v43, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s61
+; SI-NEXT: v_cvt_f16_f32_e32 v42, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s63
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v40, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v41, v0
+; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v42
+; SI-NEXT: v_lshlrev_b32_e32 v42, 16, v53
+; SI-NEXT: v_lshlrev_b32_e32 v53, 16, v7
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
; SI-NEXT: v_cvt_f32_f16_e32 v0, s72
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_cvt_f16_f32_e32 v45, v1
-; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v41
+; SI-NEXT: v_cvt_f16_f32_e32 v44, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s61
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v42, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v43, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s62
-; SI-NEXT: v_lshlrev_b32_e32 v61, 16, v45
-; SI-NEXT: v_lshlrev_b32_e32 v45, 16, v53
-; SI-NEXT: v_lshlrev_b32_e32 v56, 16, v42
-; SI-NEXT: v_lshlrev_b32_e32 v42, 16, v52
+; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
+; SI-NEXT: v_cvt_f16_f32_e32 v46, v1
+; SI-NEXT: v_lshlrev_b32_e32 v56, 16, v43
+; SI-NEXT: v_lshlrev_b32_e32 v43, 16, v52
+; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v48
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v44, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v45, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s60
-; SI-NEXT: v_lshlrev_b32_e32 v53, 16, v51
-; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v39
; SI-NEXT: v_cvt_f32_f16_e32 v1, s59
+; SI-NEXT: v_lshlrev_b32_e32 v61, 16, v46
+; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v45
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v46, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v47, v0
+; SI-NEXT: v_lshlrev_b32_e32 v45, 16, v54
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v49
; SI-NEXT: v_cvt_f32_f16_e32 v0, s58
+; SI-NEXT: v_lshlrev_b32_e32 v60, 16, v47
+; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v40
+; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v51
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v35, v1
-; SI-NEXT: v_lshlrev_b32_e32 v60, 16, v46
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v40
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v49
; SI-NEXT: v_cvt_f32_f16_e32 v1, s57
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v34, v0
@@ -217934,9 +218050,6 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_cvt_f32_f16_e32 v0, s26
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v21, v1
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v20, v0
@@ -217946,6 +218059,9 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_cvt_f32_f16_e32 v1, s14
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v18, v0
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v7
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: v_cvt_f32_f16_e32 v0, s24
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v17, v1
@@ -218001,24 +218117,21 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
-; SI-NEXT: v_cvt_f16_f32_e32 v48, v1
+; SI-NEXT: v_cvt_f16_f32_e32 v39, v1
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v50
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v39
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v38
; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v37
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v36
@@ -218028,6 +218141,9 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v7
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
@@ -218041,21 +218157,18 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v34
; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v35
-; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v44
-; SI-NEXT: v_lshlrev_b32_e32 v59, 16, v43
-; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v54
-; SI-NEXT: v_lshlrev_b32_e32 v43, 16, v16
-; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v8
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v59, 16, v44
+; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v41
+; SI-NEXT: v_lshlrev_b32_e32 v44, 16, v55
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v16
+; SI-NEXT: v_lshlrev_b32_e32 v55, 16, v8
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
@@ -218125,7 +218238,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_mov_b32_e32 v38, s4
; SI-NEXT: v_readlane_b32 s4, v62, 26
; SI-NEXT: v_mov_b32_e32 v0, s86
-; SI-NEXT: v_mov_b32_e32 v48, s4
+; SI-NEXT: v_mov_b32_e32 v39, s4
; SI-NEXT: v_readlane_b32 s4, v62, 27
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
@@ -218154,25 +218267,25 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s11
+; SI-NEXT: v_mov_b32_e32 v0, s82
; SI-NEXT: v_mov_b32_e32 v1, s4
; SI-NEXT: v_readlane_b32 s4, v62, 33
+; SI-NEXT: v_mov_b32_e32 v49, s11
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v39, s82
-; SI-NEXT: v_mov_b32_e32 v51, s81
-; SI-NEXT: v_mov_b32_e32 v49, s10
+; SI-NEXT: v_mov_b32_e32 v51, s10
+; SI-NEXT: v_mov_b32_e32 v48, s81
; SI-NEXT: v_mov_b32_e32 v53, s80
-; SI-NEXT: v_mov_b32_e32 v52, s71
-; SI-NEXT: v_mov_b32_e32 v55, s9
+; SI-NEXT: v_mov_b32_e32 v52, s9
+; SI-NEXT: v_mov_b32_e32 v55, s71
; SI-NEXT: v_mov_b32_e32 v54, s70
-; SI-NEXT: v_mov_b32_e32 v41, s69
-; SI-NEXT: v_mov_b32_e32 v40, s8
-; SI-NEXT: v_mov_b32_e32 v43, s68
-; SI-NEXT: v_mov_b32_e32 v42, s67
-; SI-NEXT: v_mov_b32_e32 v45, s66
-; SI-NEXT: v_mov_b32_e32 v44, s7
-; SI-NEXT: v_mov_b32_e32 v47, s65
-; SI-NEXT: v_mov_b32_e32 v46, s6
+; SI-NEXT: v_mov_b32_e32 v41, s8
+; SI-NEXT: v_mov_b32_e32 v40, s69
+; SI-NEXT: v_mov_b32_e32 v43, s7
+; SI-NEXT: v_mov_b32_e32 v42, s68
+; SI-NEXT: v_mov_b32_e32 v45, s6
+; SI-NEXT: v_mov_b32_e32 v44, s67
+; SI-NEXT: v_mov_b32_e32 v47, s66
+; SI-NEXT: v_mov_b32_e32 v46, s65
; SI-NEXT: v_mov_b32_e32 v57, s64
; SI-NEXT: v_mov_b32_e32 v56, s55
; SI-NEXT: v_mov_b32_e32 v59, s54
@@ -218198,7 +218311,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v5
; SI-NEXT: v_mul_f32_e32 v1, 1.0, v1
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v0
-; SI-NEXT: v_mul_f32_e32 v0, 1.0, v48
+; SI-NEXT: v_mul_f32_e32 v0, 1.0, v39
; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v1
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v6
@@ -218266,10 +218379,11 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v0
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v51
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v0
+; SI-NEXT: v_mul_f32_e32 v0, 1.0, v49
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: v_mul_f32_e32 v9, 1.0, v50
-; SI-NEXT: v_mul_f32_e32 v50, 1.0, v49
-; SI-NEXT: v_mul_f32_e32 v48, 1.0, v39
+; SI-NEXT: v_mul_f32_e32 v50, 1.0, v48
; SI-NEXT: v_mul_f32_e32 v2, 1.0, v2
; SI-NEXT: v_mul_f32_e32 v7, 1.0, v4
; SI-NEXT: v_lshr_b64 v[33:34], v[34:35], 16
@@ -218331,8 +218445,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s35, v63, 1
; SI-NEXT: v_readlane_b32 s34, v63, 0
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_mul_f32_e32 v0, 1.0, v0
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v0
+; SI-NEXT: v_mul_f32_e32 v48, 1.0, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
@@ -218408,8 +218521,8 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_mov_b32_e32 v0, v33
; SI-NEXT: v_mov_b32_e32 v1, v34
; SI-NEXT: s_or_saveexec_b64 s[4:5], -1
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -220126,8 +220239,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v34, vcc, 0x7fff, v34
; VI-NEXT: v_or_b32_e32 v35, 0x400000, v17
; VI-NEXT: v_cmp_u_f32_e32 vcc, v17, v17
-; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; VI-NEXT: v_cndmask_b32_e32 v32, v34, v35, vcc
+; VI-NEXT: v_cndmask_b32_e32 v17, v34, v35, vcc
; VI-NEXT: v_lshlrev_b32_e32 v34, 16, v18
; VI-NEXT: v_add_f32_e32 v34, 0x40c00000, v34
; VI-NEXT: v_bfe_u32 v35, v34, 16, 1
@@ -220336,7 +220448,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v40, 0x400000, v30
; VI-NEXT: v_cmp_u_f32_e32 vcc, v30, v30
; VI-NEXT: v_cndmask_b32_e32 v30, v55, v40, vcc
-; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_lshlrev_b32_e32 v55, 16, v31
; VI-NEXT: v_add_f32_e32 v55, 0x40c00000, v55
; VI-NEXT: v_bfe_u32 v40, v55, 16, 1
@@ -220368,7 +220480,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v41, vcc, 0x7fff, v41
; VI-NEXT: v_or_b32_e32 v42, 0x400000, v0
; VI-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
-; VI-NEXT: v_cndmask_b32_e32 v17, v41, v42, vcc
+; VI-NEXT: v_cndmask_b32_e32 v0, v41, v42, vcc
; VI-NEXT: v_lshlrev_b32_e32 v41, 16, v1
; VI-NEXT: v_add_f32_e32 v41, 0x40c00000, v41
; VI-NEXT: v_bfe_u32 v42, v41, 16, 1
@@ -220384,8 +220496,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v42, vcc, 0x7fff, v42
; VI-NEXT: v_or_b32_e32 v43, 0x400000, v1
; VI-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; VI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; VI-NEXT: v_cndmask_b32_e32 v16, v42, v43, vcc
+; VI-NEXT: v_cndmask_b32_e32 v1, v42, v43, vcc
; VI-NEXT: v_lshlrev_b32_e32 v42, 16, v2
; VI-NEXT: v_add_f32_e32 v42, 0x40c00000, v42
; VI-NEXT: v_bfe_u32 v43, v42, 16, 1
@@ -220408,274 +220519,273 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; VI-NEXT: v_add_u32_e32 v44, vcc, v44, v43
; VI-NEXT: v_add_u32_e32 v44, vcc, 0x7fff, v44
; VI-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
-; VI-NEXT: v_or_b32_e32 v45, 0x400000, v43
; VI-NEXT: v_cmp_u_f32_e32 vcc, v43, v43
+; VI-NEXT: v_or_b32_e32 v43, 0x400000, v43
; VI-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
-; VI-NEXT: v_cndmask_b32_e32 v43, v44, v45, vcc
+; VI-NEXT: v_cndmask_b32_e32 v43, v44, v43, vcc
; VI-NEXT: v_bfe_u32 v44, v3, 16, 1
; VI-NEXT: v_add_u32_e32 v44, vcc, v44, v3
; VI-NEXT: v_add_u32_e32 v44, vcc, 0x7fff, v44
-; VI-NEXT: v_or_b32_e32 v45, 0x400000, v3
; VI-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
-; VI-NEXT: v_cndmask_b32_e32 v3, v44, v45, vcc
+; VI-NEXT: v_or_b32_e32 v3, 0x400000, v3
+; VI-NEXT: v_cndmask_b32_e32 v3, v44, v3, vcc
; VI-NEXT: v_lshlrev_b32_e32 v44, 16, v4
; VI-NEXT: v_add_f32_e32 v44, 0x40c00000, v44
; VI-NEXT: v_bfe_u32 v45, v44, 16, 1
; VI-NEXT: v_add_u32_e32 v45, vcc, v45, v44
; VI-NEXT: v_add_u32_e32 v45, vcc, 0x7fff, v45
; VI-NEXT: v_and_b32_e32 v4, 0xffff0000, v4
-; VI-NEXT: v_or_b32_e32 v46, 0x400000, v44
; VI-NEXT: v_cmp_u_f32_e32 vcc, v44, v44
+; VI-NEXT: v_or_b32_e32 v44, 0x400000, v44
; VI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
-; VI-NEXT: v_cndmask_b32_e32 v44, v45, v46, vcc
+; VI-NEXT: v_cndmask_b32_e32 v44, v45, v44, vcc
; VI-NEXT: v_bfe_u32 v45, v4, 16, 1
; VI-NEXT: v_add_u32_e32 v45, vcc, v45, v4
; VI-NEXT: v_add_u32_e32 v45, vcc, 0x7fff, v45
-; VI-NEXT: v_or_b32_e32 v46, 0x400000, v4
; VI-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
-; VI-NEXT: v_cndmask_b32_e32 v4, v45, v46, vcc
+; VI-NEXT: v_or_b32_e32 v4, 0x400000, v4
+; VI-NEXT: v_cndmask_b32_e32 v4, v45, v4, vcc
; VI-NEXT: v_lshlrev_b32_e32 v45, 16, v5
; VI-NEXT: v_add_f32_e32 v45, 0x40c00000, v45
; VI-NEXT: v_bfe_u32 v46, v45, 16, 1
; VI-NEXT: v_add_u32_e32 v46, vcc, v46, v45
; VI-NEXT: v_add_u32_e32 v46, vcc, 0x7fff, v46
; VI-NEXT: v_and_b32_e32 v5, 0xffff0000, v5
-; VI-NEXT: v_or_b32_e32 v47, 0x400000, v45
; VI-NEXT: v_cmp_u_f32_e32 vcc, v45, v45
+; VI-NEXT: v_or_b32_e32 v45, 0x400000, v45
; VI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
-; VI-NEXT: v_cndmask_b32_e32 v45, v46, v47, vcc
+; VI-NEXT: v_cndmask_b32_e32 v45, v46, v45, vcc
; VI-NEXT: v_bfe_u32 v46, v5, 16, 1
; VI-NEXT: v_add_u32_e32 v46, vcc, v46, v5
; VI-NEXT: v_add_u32_e32 v46, vcc, 0x7fff, v46
-; VI-NEXT: v_or_b32_e32 v47, 0x400000, v5
; VI-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; VI-NEXT: v_cndmask_b32_e32 v5, v46, v47, vcc
+; VI-NEXT: v_or_b32_e32 v5, 0x400000, v5
+; VI-NEXT: v_cndmask_b32_e32 v5, v46, v5, vcc
; VI-NEXT: v_lshlrev_b32_e32 v46, 16, v6
; VI-NEXT: v_add_f32_e32 v46, 0x40c00000, v46
; VI-NEXT: v_bfe_u32 v47, v46, 16, 1
; VI-NEXT: v_add_u32_e32 v47, vcc, v47, v46
; VI-NEXT: v_add_u32_e32 v47, vcc, 0x7fff, v47
; VI-NEXT: v_and_b32_e32 v6, 0xffff0000, v6
-; VI-NEXT: v_or_b32_e32 v56, 0x400000, v46
; VI-NEXT: v_cmp_u_f32_e32 vcc, v46, v46
+; VI-NEXT: v_or_b32_e32 v46, 0x400000, v46
; VI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
-; VI-NEXT: v_cndmask_b32_e32 v46, v47, v56, vcc
+; VI-NEXT: v_cndmask_b32_e32 v46, v47, v46, vcc
; VI-NEXT: v_bfe_u32 v47, v6, 16, 1
; VI-NEXT: v_add_u32_e32 v47, vcc, v47, v6
; VI-NEXT: v_add_u32_e32 v47, vcc, 0x7fff, v47
-; VI-NEXT: v_or_b32_e32 v56, 0x400000, v6
; VI-NEXT: v_cmp_u_f32_e32 vcc, v6, v6
-; VI-NEXT: v_cndmask_b32_e32 v6, v47, v56, vcc
+; VI-NEXT: v_or_b32_e32 v6, 0x400000, v6
+; VI-NEXT: v_cndmask_b32_e32 v6, v47, v6, vcc
; VI-NEXT: v_lshlrev_b32_e32 v47, 16, v7
; VI-NEXT: v_add_f32_e32 v47, 0x40c00000, v47
; VI-NEXT: v_bfe_u32 v56, v47, 16, 1
; VI-NEXT: v_add_u32_e32 v56, vcc, v56, v47
; VI-NEXT: v_add_u32_e32 v56, vcc, 0x7fff, v56
; VI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
-; VI-NEXT: v_or_b32_e32 v57, 0x400000, v47
; VI-NEXT: v_cmp_u_f32_e32 vcc, v47, v47
+; VI-NEXT: v_or_b32_e32 v47, 0x400000, v47
; VI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; VI-NEXT: v_cndmask_b32_e32 v47, v56, v57, vcc
+; VI-NEXT: v_cndmask_b32_e32 v47, v56, v47, vcc
; VI-NEXT: v_bfe_u32 v56, v7, 16, 1
; VI-NEXT: v_add_u32_e32 v56, vcc, v56, v7
; VI-NEXT: v_add_u32_e32 v56, vcc, 0x7fff, v56
-; VI-NEXT: v_or_b32_e32 v57, 0x400000, v7
; VI-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
-; VI-NEXT: v_cndmask_b32_e32 v7, v56, v57, vcc
+; VI-NEXT: v_or_b32_e32 v7, 0x400000, v7
+; VI-NEXT: v_cndmask_b32_e32 v7, v56, v7, vcc
; VI-NEXT: v_lshlrev_b32_e32 v56, 16, v8
; VI-NEXT: v_add_f32_e32 v56, 0x40c00000, v56
; VI-NEXT: v_bfe_u32 v57, v56, 16, 1
; VI-NEXT: v_add_u32_e32 v57, vcc, v57, v56
; VI-NEXT: v_add_u32_e32 v57, vcc, 0x7fff, v57
; VI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; VI-NEXT: v_or_b32_e32 v58, 0x400000, v56
; VI-NEXT: v_cmp_u_f32_e32 vcc, v56, v56
+; VI-NEXT: v_or_b32_e32 v56, 0x400000, v56
; VI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; VI-NEXT: v_cndmask_b32_e32 v56, v57, v58, vcc
+; VI-NEXT: v_cndmask_b32_e32 v56, v57, v56, vcc
; VI-NEXT: v_bfe_u32 v57, v8, 16, 1
; VI-NEXT: v_add_u32_e32 v57, vcc, v57, v8
; VI-NEXT: v_add_u32_e32 v57, vcc, 0x7fff, v57
-; VI-NEXT: v_or_b32_e32 v58, 0x400000, v8
; VI-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; VI-NEXT: v_cndmask_b32_e32 v8, v57, v58, vcc
+; VI-NEXT: v_or_b32_e32 v8, 0x400000, v8
+; VI-NEXT: v_cndmask_b32_e32 v8, v57, v8, vcc
; VI-NEXT: v_lshlrev_b32_e32 v57, 16, v9
; VI-NEXT: v_add_f32_e32 v57, 0x40c00000, v57
; VI-NEXT: v_bfe_u32 v58, v57, 16, 1
; VI-NEXT: v_add_u32_e32 v58, vcc, v58, v57
; VI-NEXT: v_add_u32_e32 v58, vcc, 0x7fff, v58
; VI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; VI-NEXT: v_or_b32_e32 v59, 0x400000, v57
; VI-NEXT: v_cmp_u_f32_e32 vcc, v57, v57
+; VI-NEXT: v_or_b32_e32 v57, 0x400000, v57
; VI-NEXT: v_add_f32_e32 v9, 0x40c00000, v9
-; VI-NEXT: v_cndmask_b32_e32 v57, v58, v59, vcc
+; VI-NEXT: v_cndmask_b32_e32 v57, v58, v57, vcc
; VI-NEXT: v_bfe_u32 v58, v9, 16, 1
; VI-NEXT: v_add_u32_e32 v58, vcc, v58, v9
; VI-NEXT: v_add_u32_e32 v58, vcc, 0x7fff, v58
-; VI-NEXT: v_or_b32_e32 v59, 0x400000, v9
; VI-NEXT: v_cmp_u_f32_e32 vcc, v9, v9
-; VI-NEXT: v_cndmask_b32_e32 v9, v58, v59, vcc
+; VI-NEXT: v_or_b32_e32 v9, 0x400000, v9
+; VI-NEXT: v_cndmask_b32_e32 v9, v58, v9, vcc
; VI-NEXT: v_lshlrev_b32_e32 v58, 16, v10
; VI-NEXT: v_add_f32_e32 v58, 0x40c00000, v58
; VI-NEXT: v_bfe_u32 v59, v58, 16, 1
; VI-NEXT: v_add_u32_e32 v59, vcc, v59, v58
; VI-NEXT: v_add_u32_e32 v59, vcc, 0x7fff, v59
; VI-NEXT: v_and_b32_e32 v10, 0xffff0000, v10
-; VI-NEXT: v_or_b32_e32 v60, 0x400000, v58
; VI-NEXT: v_cmp_u_f32_e32 vcc, v58, v58
+; VI-NEXT: v_or_b32_e32 v58, 0x400000, v58
; VI-NEXT: v_add_f32_e32 v10, 0x40c00000, v10
-; VI-NEXT: v_cndmask_b32_e32 v58, v59, v60, vcc
+; VI-NEXT: v_cndmask_b32_e32 v58, v59, v58, vcc
; VI-NEXT: v_bfe_u32 v59, v10, 16, 1
; VI-NEXT: v_add_u32_e32 v59, vcc, v59, v10
; VI-NEXT: v_add_u32_e32 v59, vcc, 0x7fff, v59
-; VI-NEXT: v_or_b32_e32 v60, 0x400000, v10
; VI-NEXT: v_cmp_u_f32_e32 vcc, v10, v10
-; VI-NEXT: v_cndmask_b32_e32 v10, v59, v60, vcc
+; VI-NEXT: v_or_b32_e32 v10, 0x400000, v10
+; VI-NEXT: v_cndmask_b32_e32 v10, v59, v10, vcc
; VI-NEXT: v_lshlrev_b32_e32 v59, 16, v11
; VI-NEXT: v_add_f32_e32 v59, 0x40c00000, v59
; VI-NEXT: v_bfe_u32 v60, v59, 16, 1
; VI-NEXT: v_add_u32_e32 v60, vcc, v60, v59
; VI-NEXT: v_add_u32_e32 v60, vcc, 0x7fff, v60
; VI-NEXT: v_and_b32_e32 v11, 0xffff0000, v11
-; VI-NEXT: v_or_b32_e32 v61, 0x400000, v59
; VI-NEXT: v_cmp_u_f32_e32 vcc, v59, v59
+; VI-NEXT: v_or_b32_e32 v59, 0x400000, v59
; VI-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
-; VI-NEXT: v_cndmask_b32_e32 v59, v60, v61, vcc
+; VI-NEXT: v_cndmask_b32_e32 v59, v60, v59, vcc
; VI-NEXT: v_bfe_u32 v60, v11, 16, 1
; VI-NEXT: v_add_u32_e32 v60, vcc, v60, v11
; VI-NEXT: v_add_u32_e32 v60, vcc, 0x7fff, v60
-; VI-NEXT: v_or_b32_e32 v61, 0x400000, v11
; VI-NEXT: v_cmp_u_f32_e32 vcc, v11, v11
-; VI-NEXT: v_cndmask_b32_e32 v11, v60, v61, vcc
+; VI-NEXT: v_or_b32_e32 v11, 0x400000, v11
+; VI-NEXT: v_cndmask_b32_e32 v11, v60, v11, vcc
; VI-NEXT: v_lshlrev_b32_e32 v60, 16, v12
; VI-NEXT: v_add_f32_e32 v60, 0x40c00000, v60
; VI-NEXT: v_bfe_u32 v61, v60, 16, 1
; VI-NEXT: v_add_u32_e32 v61, vcc, v61, v60
; VI-NEXT: v_add_u32_e32 v61, vcc, 0x7fff, v61
; VI-NEXT: v_and_b32_e32 v12, 0xffff0000, v12
-; VI-NEXT: v_or_b32_e32 v62, 0x400000, v60
; VI-NEXT: v_cmp_u_f32_e32 vcc, v60, v60
+; VI-NEXT: v_or_b32_e32 v60, 0x400000, v60
; VI-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
-; VI-NEXT: v_cndmask_b32_e32 v60, v61, v62, vcc
+; VI-NEXT: v_cndmask_b32_e32 v60, v61, v60, vcc
; VI-NEXT: v_bfe_u32 v61, v12, 16, 1
; VI-NEXT: v_add_u32_e32 v61, vcc, v61, v12
; VI-NEXT: v_add_u32_e32 v61, vcc, 0x7fff, v61
-; VI-NEXT: v_or_b32_e32 v62, 0x400000, v12
; VI-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
-; VI-NEXT: v_cndmask_b32_e32 v12, v61, v62, vcc
+; VI-NEXT: v_or_b32_e32 v12, 0x400000, v12
+; VI-NEXT: v_cndmask_b32_e32 v12, v61, v12, vcc
; VI-NEXT: v_lshlrev_b32_e32 v61, 16, v13
; VI-NEXT: v_add_f32_e32 v61, 0x40c00000, v61
; VI-NEXT: v_bfe_u32 v62, v61, 16, 1
; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v61
; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
; VI-NEXT: v_and_b32_e32 v13, 0xffff0000, v13
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v61
; VI-NEXT: v_cmp_u_f32_e32 vcc, v61, v61
+; VI-NEXT: v_or_b32_e32 v61, 0x400000, v61
; VI-NEXT: v_add_f32_e32 v13, 0x40c00000, v13
-; VI-NEXT: v_cndmask_b32_e32 v61, v62, v63, vcc
+; VI-NEXT: v_cndmask_b32_e32 v61, v62, v61, vcc
; VI-NEXT: v_bfe_u32 v62, v13, 16, 1
; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v13
; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v13
; VI-NEXT: v_cmp_u_f32_e32 vcc, v13, v13
-; VI-NEXT: v_cndmask_b32_e32 v13, v62, v63, vcc
+; VI-NEXT: v_or_b32_e32 v13, 0x400000, v13
+; VI-NEXT: v_cndmask_b32_e32 v13, v62, v13, vcc
; VI-NEXT: v_lshlrev_b32_e32 v62, 16, v14
; VI-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
; VI-NEXT: v_bfe_u32 v63, v62, 16, 1
; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v62
-; VI-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
-; VI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
-; VI-NEXT: v_or_b32_e32 v0, 0x400000, v62
+; VI-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; VI-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; VI-NEXT: v_bfe_u32 v62, v14, 16, 1
-; VI-NEXT: v_cndmask_b32_e32 v1, v63, v0, vcc
-; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v14
-; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v14
+; VI-NEXT: v_or_b32_e32 v62, 0x400000, v62
+; VI-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; VI-NEXT: v_cndmask_b32_e32 v62, v63, v62, vcc
+; VI-NEXT: v_bfe_u32 v63, v14, 16, 1
+; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v14
+; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
; VI-NEXT: v_cmp_u_f32_e32 vcc, v14, v14
-; VI-NEXT: v_cndmask_b32_e32 v14, v62, v63, vcc
-; VI-NEXT: v_lshlrev_b32_e32 v62, 16, v15
-; VI-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
-; VI-NEXT: v_bfe_u32 v63, v62, 16, 1
-; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v62
+; VI-NEXT: v_or_b32_e32 v14, 0x400000, v14
+; VI-NEXT: v_cndmask_b32_e32 v14, v63, v14, vcc
+; VI-NEXT: v_lshlrev_b32_e32 v63, 16, v15
+; VI-NEXT: v_add_f32_e32 v63, 0x40c00000, v63
+; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; VI-NEXT: v_bfe_u32 v32, v63, 16, 1
+; VI-NEXT: v_add_u32_e32 v32, vcc, v32, v63
+; VI-NEXT: v_add_u32_e32 v32, vcc, 0x7fff, v32
; VI-NEXT: v_and_b32_e32 v15, 0xffff0000, v15
-; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
+; VI-NEXT: v_cmp_u_f32_e32 vcc, v63, v63
+; VI-NEXT: v_or_b32_e32 v63, 0x400000, v63
; VI-NEXT: v_add_f32_e32 v15, 0x40c00000, v15
-; VI-NEXT: v_or_b32_e32 v0, 0x400000, v62
-; VI-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; VI-NEXT: v_bfe_u32 v62, v15, 16, 1
-; VI-NEXT: v_cndmask_b32_e32 v0, v63, v0, vcc
-; VI-NEXT: v_add_u32_e32 v62, vcc, v62, v15
-; VI-NEXT: v_add_u32_e32 v62, vcc, 0x7fff, v62
-; VI-NEXT: v_or_b32_e32 v63, 0x400000, v15
+; VI-NEXT: v_cndmask_b32_e32 v32, v32, v63, vcc
+; VI-NEXT: v_bfe_u32 v63, v15, 16, 1
+; VI-NEXT: v_add_u32_e32 v63, vcc, v63, v15
+; VI-NEXT: v_add_u32_e32 v63, vcc, 0x7fff, v63
; VI-NEXT: v_cmp_u_f32_e32 vcc, v15, v15
-; VI-NEXT: v_cndmask_b32_e32 v15, v62, v63, vcc
+; VI-NEXT: v_or_b32_e32 v15, 0x400000, v15
+; VI-NEXT: v_cndmask_b32_e32 v15, v63, v15, vcc
; VI-NEXT: v_lshrrev_b32_e32 v15, 16, v15
-; VI-NEXT: v_alignbit_b32 v15, v15, v0, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v14
-; VI-NEXT: v_alignbit_b32 v14, v0, v1, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v13
-; VI-NEXT: v_alignbit_b32 v13, v0, v61, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v12
-; VI-NEXT: v_alignbit_b32 v12, v0, v60, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v11
-; VI-NEXT: v_alignbit_b32 v11, v0, v59, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v10
-; VI-NEXT: v_alignbit_b32 v10, v0, v58, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v9
-; VI-NEXT: v_alignbit_b32 v9, v0, v57, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v8
-; VI-NEXT: v_alignbit_b32 v8, v0, v56, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v7
-; VI-NEXT: v_alignbit_b32 v7, v0, v47, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v6
-; VI-NEXT: v_alignbit_b32 v6, v0, v46, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v5
-; VI-NEXT: v_alignbit_b32 v5, v0, v45, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v4
-; VI-NEXT: v_alignbit_b32 v4, v0, v44, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v3
-; VI-NEXT: v_alignbit_b32 v3, v0, v43, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v2
-; VI-NEXT: v_alignbit_b32 v2, v0, v42, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v31
-; VI-NEXT: v_alignbit_b32 v31, v16, v55, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v30
-; VI-NEXT: v_alignbit_b32 v30, v16, v54, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v29
-; VI-NEXT: v_alignbit_b32 v29, v16, v53, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v28
-; VI-NEXT: v_alignbit_b32 v28, v16, v52, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v27
-; VI-NEXT: v_alignbit_b32 v27, v16, v51, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v26
-; VI-NEXT: v_alignbit_b32 v26, v16, v50, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v25
-; VI-NEXT: v_alignbit_b32 v25, v16, v49, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v24
-; VI-NEXT: v_alignbit_b32 v24, v16, v48, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v23
-; VI-NEXT: v_alignbit_b32 v23, v16, v39, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v22
-; VI-NEXT: v_alignbit_b32 v22, v16, v38, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v21
-; VI-NEXT: v_alignbit_b32 v21, v16, v37, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v20
-; VI-NEXT: v_alignbit_b32 v20, v16, v36, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v19
-; VI-NEXT: v_alignbit_b32 v19, v16, v35, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v18
-; VI-NEXT: v_alignbit_b32 v18, v16, v34, 16
-; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v32
-; VI-NEXT: v_alignbit_b32 v1, v0, v41, 16
-; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v17
-; VI-NEXT: v_alignbit_b32 v17, v16, v33, 16
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; VI-NEXT: v_alignbit_b32 v15, v15, v32, 16
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; VI-NEXT: v_alignbit_b32 v0, v0, v40, 16
-; VI-NEXT: s_waitcnt vmcnt(1)
+; VI-NEXT: v_lshrrev_b32_e32 v14, 16, v14
+; VI-NEXT: v_lshrrev_b32_e32 v13, 16, v13
+; VI-NEXT: v_lshrrev_b32_e32 v12, 16, v12
+; VI-NEXT: v_lshrrev_b32_e32 v11, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v10, 16, v10
+; VI-NEXT: v_lshrrev_b32_e32 v9, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v8, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v7, 16, v7
+; VI-NEXT: v_lshrrev_b32_e32 v6, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v5, 16, v5
+; VI-NEXT: v_lshrrev_b32_e32 v4, 16, v4
+; VI-NEXT: v_lshrrev_b32_e32 v3, 16, v3
+; VI-NEXT: v_lshrrev_b32_e32 v2, 16, v2
+; VI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
+; VI-NEXT: v_lshrrev_b32_e32 v0, 16, v0
+; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v31
+; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v30
+; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v29
+; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v28
+; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v27
+; VI-NEXT: v_lshrrev_b32_e32 v26, 16, v26
+; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v25
+; VI-NEXT: v_lshrrev_b32_e32 v24, 16, v24
+; VI-NEXT: v_lshrrev_b32_e32 v23, 16, v23
+; VI-NEXT: v_lshrrev_b32_e32 v22, 16, v22
+; VI-NEXT: v_lshrrev_b32_e32 v21, 16, v21
+; VI-NEXT: v_lshrrev_b32_e32 v20, 16, v20
+; VI-NEXT: v_lshrrev_b32_e32 v19, 16, v19
+; VI-NEXT: v_lshrrev_b32_e32 v18, 16, v18
+; VI-NEXT: v_lshrrev_b32_e32 v17, 16, v17
; VI-NEXT: v_lshrrev_b32_e32 v16, 16, v16
+; VI-NEXT: v_alignbit_b32 v14, v14, v62, 16
+; VI-NEXT: v_alignbit_b32 v13, v13, v61, 16
+; VI-NEXT: v_alignbit_b32 v12, v12, v60, 16
+; VI-NEXT: v_alignbit_b32 v11, v11, v59, 16
+; VI-NEXT: v_alignbit_b32 v10, v10, v58, 16
+; VI-NEXT: v_alignbit_b32 v9, v9, v57, 16
+; VI-NEXT: v_alignbit_b32 v8, v8, v56, 16
+; VI-NEXT: v_alignbit_b32 v7, v7, v47, 16
+; VI-NEXT: v_alignbit_b32 v6, v6, v46, 16
+; VI-NEXT: v_alignbit_b32 v5, v5, v45, 16
+; VI-NEXT: v_alignbit_b32 v4, v4, v44, 16
+; VI-NEXT: v_alignbit_b32 v3, v3, v43, 16
+; VI-NEXT: v_alignbit_b32 v2, v2, v42, 16
+; VI-NEXT: v_alignbit_b32 v1, v1, v41, 16
+; VI-NEXT: v_alignbit_b32 v0, v0, v40, 16
+; VI-NEXT: v_alignbit_b32 v31, v31, v55, 16
+; VI-NEXT: v_alignbit_b32 v30, v30, v54, 16
+; VI-NEXT: v_alignbit_b32 v29, v29, v53, 16
+; VI-NEXT: v_alignbit_b32 v28, v28, v52, 16
+; VI-NEXT: v_alignbit_b32 v27, v27, v51, 16
+; VI-NEXT: v_alignbit_b32 v26, v26, v50, 16
+; VI-NEXT: v_alignbit_b32 v25, v25, v49, 16
+; VI-NEXT: v_alignbit_b32 v24, v24, v48, 16
+; VI-NEXT: v_alignbit_b32 v23, v23, v39, 16
+; VI-NEXT: v_alignbit_b32 v22, v22, v38, 16
+; VI-NEXT: v_alignbit_b32 v21, v21, v37, 16
+; VI-NEXT: v_alignbit_b32 v20, v20, v36, 16
+; VI-NEXT: v_alignbit_b32 v19, v19, v35, 16
+; VI-NEXT: v_alignbit_b32 v18, v18, v34, 16
+; VI-NEXT: v_alignbit_b32 v17, v17, v33, 16
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_alignbit_b32 v16, v16, v32, 16
; VI-NEXT: .LBB104_2: ; %end
@@ -220755,8 +220865,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_add3_u32 v34, v34, v17, s6
; GFX9-NEXT: v_or_b32_e32 v35, 0x400000, v17
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v17, v17
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; GFX9-NEXT: v_cndmask_b32_e32 v32, v34, v35, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v17, v34, v35, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v34, 16, v18
; GFX9-NEXT: v_add_f32_e32 v34, 0x40c00000, v34
; GFX9-NEXT: v_bfe_u32 v35, v34, 16, 1
@@ -220939,7 +221048,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_or_b32_e32 v40, 0x400000, v30
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v30, v30
; GFX9-NEXT: v_cndmask_b32_e32 v30, v55, v40, vcc
-; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_lshlrev_b32_e32 v55, 16, v31
; GFX9-NEXT: v_add_f32_e32 v55, 0x40c00000, v55
; GFX9-NEXT: v_bfe_u32 v40, v55, 16, 1
@@ -220967,7 +221076,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_add3_u32 v41, v41, v0, s6
; GFX9-NEXT: v_or_b32_e32 v42, 0x400000, v0
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
-; GFX9-NEXT: v_cndmask_b32_e32 v17, v41, v42, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v0, v41, v42, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v41, 16, v1
; GFX9-NEXT: v_add_f32_e32 v41, 0x40c00000, v41
; GFX9-NEXT: v_bfe_u32 v42, v41, 16, 1
@@ -220981,8 +221090,7 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_add3_u32 v42, v42, v1, s6
; GFX9-NEXT: v_or_b32_e32 v43, 0x400000, v1
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; GFX9-NEXT: v_cndmask_b32_e32 v16, v42, v43, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v1, v42, v43, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v42, 16, v2
; GFX9-NEXT: v_add_f32_e32 v42, 0x40c00000, v42
; GFX9-NEXT: v_bfe_u32 v43, v42, 16, 1
@@ -221002,191 +221110,188 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_bfe_u32 v44, v43, 16, 1
; GFX9-NEXT: v_and_b32_e32 v3, 0xffff0000, v3
; GFX9-NEXT: v_add3_u32 v44, v44, v43, s6
-; GFX9-NEXT: v_or_b32_e32 v45, 0x400000, v43
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v43, v43
+; GFX9-NEXT: v_or_b32_e32 v43, 0x400000, v43
; GFX9-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
-; GFX9-NEXT: v_cndmask_b32_e32 v43, v44, v45, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v43, v44, v43, vcc
; GFX9-NEXT: v_bfe_u32 v44, v3, 16, 1
; GFX9-NEXT: v_add3_u32 v44, v44, v3, s6
-; GFX9-NEXT: v_or_b32_e32 v45, 0x400000, v3
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
-; GFX9-NEXT: v_cndmask_b32_e32 v3, v44, v45, vcc
+; GFX9-NEXT: v_or_b32_e32 v3, 0x400000, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v44, v3, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v44, 16, v4
; GFX9-NEXT: v_add_f32_e32 v44, 0x40c00000, v44
; GFX9-NEXT: v_bfe_u32 v45, v44, 16, 1
; GFX9-NEXT: v_and_b32_e32 v4, 0xffff0000, v4
; GFX9-NEXT: v_add3_u32 v45, v45, v44, s6
-; GFX9-NEXT: v_or_b32_e32 v46, 0x400000, v44
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v44, v44
+; GFX9-NEXT: v_or_b32_e32 v44, 0x400000, v44
; GFX9-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
-; GFX9-NEXT: v_cndmask_b32_e32 v44, v45, v46, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v44, v45, v44, vcc
; GFX9-NEXT: v_bfe_u32 v45, v4, 16, 1
; GFX9-NEXT: v_add3_u32 v45, v45, v4, s6
-; GFX9-NEXT: v_or_b32_e32 v46, 0x400000, v4
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
-; GFX9-NEXT: v_cndmask_b32_e32 v4, v45, v46, vcc
+; GFX9-NEXT: v_or_b32_e32 v4, 0x400000, v4
+; GFX9-NEXT: v_cndmask_b32_e32 v4, v45, v4, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v45, 16, v5
; GFX9-NEXT: v_add_f32_e32 v45, 0x40c00000, v45
; GFX9-NEXT: v_bfe_u32 v46, v45, 16, 1
; GFX9-NEXT: v_and_b32_e32 v5, 0xffff0000, v5
; GFX9-NEXT: v_add3_u32 v46, v46, v45, s6
-; GFX9-NEXT: v_or_b32_e32 v47, 0x400000, v45
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v45, v45
+; GFX9-NEXT: v_or_b32_e32 v45, 0x400000, v45
; GFX9-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v45, v46, v47, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v45, v46, v45, vcc
; GFX9-NEXT: v_bfe_u32 v46, v5, 16, 1
; GFX9-NEXT: v_add3_u32 v46, v46, v5, s6
-; GFX9-NEXT: v_or_b32_e32 v47, 0x400000, v5
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v46, v47, vcc
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v5
+; GFX9-NEXT: v_cndmask_b32_e32 v5, v46, v5, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v46, 16, v6
; GFX9-NEXT: v_add_f32_e32 v46, 0x40c00000, v46
; GFX9-NEXT: v_bfe_u32 v47, v46, 16, 1
; GFX9-NEXT: v_and_b32_e32 v6, 0xffff0000, v6
; GFX9-NEXT: v_add3_u32 v47, v47, v46, s6
-; GFX9-NEXT: v_or_b32_e32 v56, 0x400000, v46
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v46, v46
+; GFX9-NEXT: v_or_b32_e32 v46, 0x400000, v46
; GFX9-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
-; GFX9-NEXT: v_cndmask_b32_e32 v46, v47, v56, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v46, v47, v46, vcc
; GFX9-NEXT: v_bfe_u32 v47, v6, 16, 1
; GFX9-NEXT: v_add3_u32 v47, v47, v6, s6
-; GFX9-NEXT: v_or_b32_e32 v56, 0x400000, v6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v6, v6
-; GFX9-NEXT: v_cndmask_b32_e32 v6, v47, v56, vcc
+; GFX9-NEXT: v_or_b32_e32 v6, 0x400000, v6
+; GFX9-NEXT: v_cndmask_b32_e32 v6, v47, v6, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v47, 16, v7
; GFX9-NEXT: v_add_f32_e32 v47, 0x40c00000, v47
; GFX9-NEXT: v_bfe_u32 v56, v47, 16, 1
; GFX9-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
; GFX9-NEXT: v_add3_u32 v56, v56, v47, s6
-; GFX9-NEXT: v_or_b32_e32 v57, 0x400000, v47
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v47, v47
+; GFX9-NEXT: v_or_b32_e32 v47, 0x400000, v47
; GFX9-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; GFX9-NEXT: v_cndmask_b32_e32 v47, v56, v57, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v47, v56, v47, vcc
; GFX9-NEXT: v_bfe_u32 v56, v7, 16, 1
; GFX9-NEXT: v_add3_u32 v56, v56, v7, s6
-; GFX9-NEXT: v_or_b32_e32 v57, 0x400000, v7
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
-; GFX9-NEXT: v_cndmask_b32_e32 v7, v56, v57, vcc
+; GFX9-NEXT: v_or_b32_e32 v7, 0x400000, v7
+; GFX9-NEXT: v_cndmask_b32_e32 v7, v56, v7, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v56, 16, v8
; GFX9-NEXT: v_add_f32_e32 v56, 0x40c00000, v56
; GFX9-NEXT: v_bfe_u32 v57, v56, 16, 1
; GFX9-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
; GFX9-NEXT: v_add3_u32 v57, v57, v56, s6
-; GFX9-NEXT: v_or_b32_e32 v58, 0x400000, v56
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v56, v56
+; GFX9-NEXT: v_or_b32_e32 v56, 0x400000, v56
; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v56, v57, v58, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v56, v57, v56, vcc
; GFX9-NEXT: v_bfe_u32 v57, v8, 16, 1
; GFX9-NEXT: v_add3_u32 v57, v57, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v58, 0x400000, v8
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v8, v57, v58, vcc
+; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v8
+; GFX9-NEXT: v_cndmask_b32_e32 v8, v57, v8, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v57, 16, v9
; GFX9-NEXT: v_add_f32_e32 v57, 0x40c00000, v57
; GFX9-NEXT: v_bfe_u32 v58, v57, 16, 1
; GFX9-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
; GFX9-NEXT: v_add3_u32 v58, v58, v57, s6
-; GFX9-NEXT: v_or_b32_e32 v59, 0x400000, v57
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v57, v57
+; GFX9-NEXT: v_or_b32_e32 v57, 0x400000, v57
; GFX9-NEXT: v_add_f32_e32 v9, 0x40c00000, v9
-; GFX9-NEXT: v_cndmask_b32_e32 v57, v58, v59, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v57, v58, v57, vcc
; GFX9-NEXT: v_bfe_u32 v58, v9, 16, 1
; GFX9-NEXT: v_add3_u32 v58, v58, v9, s6
-; GFX9-NEXT: v_or_b32_e32 v59, 0x400000, v9
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v9, v9
-; GFX9-NEXT: v_cndmask_b32_e32 v9, v58, v59, vcc
+; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v9
+; GFX9-NEXT: v_cndmask_b32_e32 v9, v58, v9, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v58, 16, v10
; GFX9-NEXT: v_add_f32_e32 v58, 0x40c00000, v58
; GFX9-NEXT: v_bfe_u32 v59, v58, 16, 1
; GFX9-NEXT: v_and_b32_e32 v10, 0xffff0000, v10
; GFX9-NEXT: v_add3_u32 v59, v59, v58, s6
-; GFX9-NEXT: v_or_b32_e32 v60, 0x400000, v58
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v58, v58
+; GFX9-NEXT: v_or_b32_e32 v58, 0x400000, v58
; GFX9-NEXT: v_add_f32_e32 v10, 0x40c00000, v10
-; GFX9-NEXT: v_cndmask_b32_e32 v58, v59, v60, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v58, v59, v58, vcc
; GFX9-NEXT: v_bfe_u32 v59, v10, 16, 1
; GFX9-NEXT: v_add3_u32 v59, v59, v10, s6
-; GFX9-NEXT: v_or_b32_e32 v60, 0x400000, v10
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v10, v10
-; GFX9-NEXT: v_cndmask_b32_e32 v10, v59, v60, vcc
+; GFX9-NEXT: v_or_b32_e32 v10, 0x400000, v10
+; GFX9-NEXT: v_cndmask_b32_e32 v10, v59, v10, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v59, 16, v11
; GFX9-NEXT: v_add_f32_e32 v59, 0x40c00000, v59
; GFX9-NEXT: v_bfe_u32 v60, v59, 16, 1
; GFX9-NEXT: v_and_b32_e32 v11, 0xffff0000, v11
; GFX9-NEXT: v_add3_u32 v60, v60, v59, s6
-; GFX9-NEXT: v_or_b32_e32 v61, 0x400000, v59
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v59, v59
+; GFX9-NEXT: v_or_b32_e32 v59, 0x400000, v59
; GFX9-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
-; GFX9-NEXT: v_cndmask_b32_e32 v59, v60, v61, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v59, v60, v59, vcc
; GFX9-NEXT: v_bfe_u32 v60, v11, 16, 1
; GFX9-NEXT: v_add3_u32 v60, v60, v11, s6
-; GFX9-NEXT: v_or_b32_e32 v61, 0x400000, v11
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v11, v11
-; GFX9-NEXT: v_cndmask_b32_e32 v11, v60, v61, vcc
+; GFX9-NEXT: v_or_b32_e32 v11, 0x400000, v11
+; GFX9-NEXT: v_cndmask_b32_e32 v11, v60, v11, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v60, 16, v12
; GFX9-NEXT: v_add_f32_e32 v60, 0x40c00000, v60
; GFX9-NEXT: v_bfe_u32 v61, v60, 16, 1
; GFX9-NEXT: v_and_b32_e32 v12, 0xffff0000, v12
; GFX9-NEXT: v_add3_u32 v61, v61, v60, s6
-; GFX9-NEXT: v_or_b32_e32 v62, 0x400000, v60
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v60, v60
+; GFX9-NEXT: v_or_b32_e32 v60, 0x400000, v60
; GFX9-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
-; GFX9-NEXT: v_cndmask_b32_e32 v60, v61, v62, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v60, v61, v60, vcc
; GFX9-NEXT: v_bfe_u32 v61, v12, 16, 1
; GFX9-NEXT: v_add3_u32 v61, v61, v12, s6
-; GFX9-NEXT: v_or_b32_e32 v62, 0x400000, v12
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
-; GFX9-NEXT: v_cndmask_b32_e32 v12, v61, v62, vcc
+; GFX9-NEXT: v_or_b32_e32 v12, 0x400000, v12
+; GFX9-NEXT: v_cndmask_b32_e32 v12, v61, v12, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v61, 16, v13
; GFX9-NEXT: v_add_f32_e32 v61, 0x40c00000, v61
; GFX9-NEXT: v_bfe_u32 v62, v61, 16, 1
; GFX9-NEXT: v_and_b32_e32 v13, 0xffff0000, v13
; GFX9-NEXT: v_add3_u32 v62, v62, v61, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v61
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v61, v61
+; GFX9-NEXT: v_or_b32_e32 v61, 0x400000, v61
; GFX9-NEXT: v_add_f32_e32 v13, 0x40c00000, v13
-; GFX9-NEXT: v_cndmask_b32_e32 v61, v62, v63, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v61, v62, v61, vcc
; GFX9-NEXT: v_bfe_u32 v62, v13, 16, 1
; GFX9-NEXT: v_add3_u32 v62, v62, v13, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v13
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v13, v13
-; GFX9-NEXT: v_cndmask_b32_e32 v13, v62, v63, vcc
+; GFX9-NEXT: v_or_b32_e32 v13, 0x400000, v13
+; GFX9-NEXT: v_cndmask_b32_e32 v13, v62, v13, vcc
; GFX9-NEXT: v_lshlrev_b32_e32 v62, 16, v14
; GFX9-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
-; GFX9-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; GFX9-NEXT: v_bfe_u32 v63, v62, 16, 1
-; GFX9-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; GFX9-NEXT: v_and_b32_e32 v14, 0xffff0000, v14
; GFX9-NEXT: v_add3_u32 v63, v63, v62, s6
-; GFX9-NEXT: v_or_b32_e32 v0, 0x400000, v62
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; GFX9-NEXT: v_bfe_u32 v62, v14, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v1, v63, v0, vcc
-; GFX9-NEXT: v_add3_u32 v62, v62, v14, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v14
+; GFX9-NEXT: v_or_b32_e32 v62, 0x400000, v62
+; GFX9-NEXT: v_add_f32_e32 v14, 0x40c00000, v14
+; GFX9-NEXT: v_cndmask_b32_e32 v62, v63, v62, vcc
+; GFX9-NEXT: v_bfe_u32 v63, v14, 16, 1
+; GFX9-NEXT: v_add3_u32 v63, v63, v14, s6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v14, v14
-; GFX9-NEXT: v_cndmask_b32_e32 v14, v62, v63, vcc
-; GFX9-NEXT: v_lshlrev_b32_e32 v62, 16, v15
-; GFX9-NEXT: v_add_f32_e32 v62, 0x40c00000, v62
+; GFX9-NEXT: v_or_b32_e32 v14, 0x400000, v14
+; GFX9-NEXT: v_cndmask_b32_e32 v14, v63, v14, vcc
+; GFX9-NEXT: v_lshlrev_b32_e32 v63, 16, v15
+; GFX9-NEXT: v_add_f32_e32 v63, 0x40c00000, v63
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; GFX9-NEXT: v_bfe_u32 v32, v63, 16, 1
; GFX9-NEXT: v_and_b32_e32 v15, 0xffff0000, v15
-; GFX9-NEXT: v_bfe_u32 v63, v62, 16, 1
+; GFX9-NEXT: v_add3_u32 v32, v32, v63, s6
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v63, v63
+; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v63
; GFX9-NEXT: v_add_f32_e32 v15, 0x40c00000, v15
-; GFX9-NEXT: v_add3_u32 v63, v63, v62, s6
-; GFX9-NEXT: v_or_b32_e32 v0, 0x400000, v62
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v62, v62
-; GFX9-NEXT: v_bfe_u32 v62, v15, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v0, v63, v0, vcc
-; GFX9-NEXT: v_add3_u32 v62, v62, v15, s6
-; GFX9-NEXT: v_or_b32_e32 v63, 0x400000, v15
+; GFX9-NEXT: v_cndmask_b32_e32 v32, v32, v63, vcc
+; GFX9-NEXT: v_bfe_u32 v63, v15, 16, 1
+; GFX9-NEXT: v_add3_u32 v63, v63, v15, s6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v15, v15
-; GFX9-NEXT: v_cndmask_b32_e32 v15, v62, v63, vcc
+; GFX9-NEXT: v_or_b32_e32 v15, 0x400000, v15
+; GFX9-NEXT: v_cndmask_b32_e32 v15, v63, v15, vcc
; GFX9-NEXT: s_mov_b32 s6, 0x7060302
-; GFX9-NEXT: v_perm_b32 v15, v15, v0, s6
-; GFX9-NEXT: v_perm_b32 v14, v14, v1, s6
-; GFX9-NEXT: v_perm_b32 v1, v16, v41, s6
-; GFX9-NEXT: v_perm_b32 v0, v17, v40, s6
-; GFX9-NEXT: v_perm_b32 v17, v32, v33, s6
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v15, v15, v32, s6
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; GFX9-NEXT: v_perm_b32 v14, v14, v62, s6
; GFX9-NEXT: v_perm_b32 v13, v13, v61, s6
; GFX9-NEXT: v_perm_b32 v12, v12, v60, s6
; GFX9-NEXT: v_perm_b32 v11, v11, v59, s6
@@ -221199,6 +221304,8 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v4, v4, v44, s6
; GFX9-NEXT: v_perm_b32 v3, v3, v43, s6
; GFX9-NEXT: v_perm_b32 v2, v2, v42, s6
+; GFX9-NEXT: v_perm_b32 v1, v1, v41, s6
+; GFX9-NEXT: v_perm_b32 v0, v0, v40, s6
; GFX9-NEXT: v_perm_b32 v31, v31, v55, s6
; GFX9-NEXT: v_perm_b32 v30, v30, v54, s6
; GFX9-NEXT: v_perm_b32 v29, v29, v53, s6
@@ -221213,8 +221320,9 @@ define <64 x i16> @bitcast_v64bf16_to_v64i16(<64 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v20, v20, v36, s6
; GFX9-NEXT: v_perm_b32 v19, v19, v35, s6
; GFX9-NEXT: v_perm_b32 v18, v18, v34, s6
+; GFX9-NEXT: v_perm_b32 v17, v17, v33, s6
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_perm_b32 v16, v32, v16, s6
+; GFX9-NEXT: v_perm_b32 v16, v16, v32, s6
; GFX9-NEXT: .LBB104_2: ; %end
; GFX9-NEXT: s_or_b64 exec, exec, s[4:5]
; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
@@ -222280,7 +222388,7 @@ define inreg <64 x i16> @bitcast_v64bf16_to_v64i16_scalar(<64 x bfloat> inreg %a
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_or_saveexec_b64 s[4:5], -1
-; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:588 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
@@ -222302,10 +222410,11 @@ define inreg <64 x i16> @bitcast_v64bf16_to_v64i16_scalar(<64 x bfloat> inreg %a
; SI-NEXT: v_writelane_b32 v63, s36, 2
; SI-NEXT: v_writelane_b32 v63, s30, 3
; SI-NEXT: v_writelane_b32 v63, s31, 4
-; SI-NEXT: s_and_b32 s14, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s24, 16
-; SI-NEXT: s_and_b32 s24, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s23, s23, 16
+; SI-NEXT: s_lshl_b32 s9, s27, 16
+; SI-NEXT: s_lshl_b32 s5, s29, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v1
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s9
; SI-NEXT: v_readfirstlane_b32 s42, v17
; SI-NEXT: v_readfirstlane_b32 s44, v16
; SI-NEXT: v_readfirstlane_b32 s46, v15
@@ -222322,18 +222431,23 @@ define inreg <64 x i16> @bitcast_v64bf16_to_v64i16_scalar(<64 x bfloat> inreg %a
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: s_and_b32 s31, s30, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s5
; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s5, s29, 16
; SI-NEXT: s_and_b32 s6, s28, 0xffff0000
; SI-NEXT: s_lshl_b32 s7, s28, 16
; SI-NEXT: s_and_b32 s8, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s27, 16
; SI-NEXT: s_and_b32 s10, s26, 0xffff0000
; SI-NEXT: s_lshl_b32 s11, s26, 16
; SI-NEXT: s_and_b32 s12, s25, 0xffff0000
; SI-NEXT: s_lshl_b32 s13, s25, 16
+; SI-NEXT: s_and_b32 s14, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s24, 16
+; SI-NEXT: s_and_b32 s24, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s23, s23, 16
; SI-NEXT: s_and_b32 s25, s22, 0xffff0000
; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_and_b32 s26, s21, 0xffff0000
@@ -222380,1095 +222494,1220 @@ define inreg <64 x i16> @bitcast_v64bf16_to_v64i16_scalar(<64 x bfloat> inreg %a
; SI-NEXT: s_lshl_b32 s94, s94, 16
; SI-NEXT: s_and_b32 vcc_hi, vcc_lo, 0xffff0000
; SI-NEXT: s_lshl_b32 vcc_lo, vcc_lo, 16
-; SI-NEXT: s_and_b32 s31, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s30, s30, 16
; SI-NEXT: s_and_b32 s35, s34, 0xffff0000
; SI-NEXT: s_lshl_b32 s34, s34, 16
; SI-NEXT: v_readfirstlane_b32 s36, v18
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s23
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s30
; SI-NEXT: s_cmp_lg_u32 s36, 0
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s16
-; SI-NEXT: v_mul_f32_e64 v51, 1.0, s41
-; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: v_mul_f32_e64 v57, 1.0, s17
-; SI-NEXT: v_mul_f32_e64 v42, 1.0, s40
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s18
-; SI-NEXT: v_mul_f32_e64 v48, 1.0, s29
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_mul_f32_e64 v60, 1.0, s19
+; SI-NEXT: v_mul_f32_e64 v52, 1.0, s16
+; SI-NEXT: v_mul_f32_e64 v54, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v42, 1.0, s17
+; SI-NEXT: v_mul_f32_e64 v40, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s18
+; SI-NEXT: v_mul_f32_e64 v34, 1.0, s29
+; SI-NEXT: v_mul_f32_e64 v47, 1.0, s19
; SI-NEXT: v_mul_f32_e64 v43, 1.0, s28
-; SI-NEXT: v_mul_f32_e64 v33, 1.0, s20
-; SI-NEXT: v_mul_f32_e64 v41, 1.0, s27
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mul_f32_e64 v61, 1.0, s21
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s20
+; SI-NEXT: v_mul_f32_e64 v55, 1.0, s27
+; SI-NEXT: v_mul_f32_e64 v57, 1.0, s21
; SI-NEXT: v_mul_f32_e64 v44, 1.0, s26
; SI-NEXT: v_mul_f32_e64 v28, 1.0, s22
-; SI-NEXT: v_mul_f32_e64 v37, 1.0, s25
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v36, 1.0, s25
+; SI-NEXT: v_mul_f32_e64 v60, 1.0, s23
; SI-NEXT: v_mul_f32_e64 v45, 1.0, s24
-; SI-NEXT: v_mul_f32_e64 v36, 1.0, s15
-; SI-NEXT: v_mul_f32_e64 v55, 1.0, s14
-; SI-NEXT: v_mul_f32_e64 v53, 1.0, s13
-; SI-NEXT: v_mul_f32_e64 v47, 1.0, s12
-; SI-NEXT: v_mul_f32_e64 v50, 1.0, s11
-; SI-NEXT: v_mul_f32_e64 v52, 1.0, s10
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
-; SI-NEXT: v_mul_f32_e64 v56, 1.0, s8
-; SI-NEXT: v_mul_f32_e64 v46, 1.0, s7
-; SI-NEXT: v_mul_f32_e64 v49, 1.0, s6
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s5
-; SI-NEXT: v_mul_f32_e64 v58, 1.0, s4
-; SI-NEXT: v_mul_f32_e64 v34, 1.0, s34
-; SI-NEXT: v_mul_f32_e64 v38, 1.0, s35
-; SI-NEXT: v_mul_f32_e64 v39, 1.0, s30
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mul_f32_e64 v62, 1.0, s31
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, vcc_lo
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, vcc_hi
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s94
-; SI-NEXT: v_mul_f32_e64 v54, 1.0, s95
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s92
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s93
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s90
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s91
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s88
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s89
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s78
-; SI-NEXT: v_mul_f32_e64 v32, 1.0, s79
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s76
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s77
-; SI-NEXT: v_mul_f32_e64 v35, 1.0, s74
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s75
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s72
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s73
-; SI-NEXT: v_mul_f32_e64 v59, 1.0, s62
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s63
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s60
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s61
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s58
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s59
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s56
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s57
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
-; SI-NEXT: v_mul_f32_e64 v40, 1.0, s47
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s44
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s45
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s42
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s43
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:408 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v61, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v53, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v62, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v58, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v59, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v48, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s4
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s34
+; SI-NEXT: v_mul_f32_e64 v39, 1.0, s35
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:412 ; 4-byte Folded Spill
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s31
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, vcc_lo
+; SI-NEXT: v_mul_f32_e64 v38, 1.0, vcc_hi
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v35, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v37, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v32, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v46, 1.0, s78
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s77
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v49, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v41, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v51, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v33, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v56, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v50, 1.0, s43
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:416 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:420 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:424 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:428 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:432 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:436 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:440 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:444 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v19, off, s[0:3], s32 offset:448 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:452 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:456 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:460 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:464 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:468 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:472 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:476 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:480 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:484 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:488 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:492 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:496 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:500 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:504 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:516 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:520 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:524 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:528 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:532 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:536 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:540 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:544 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:548 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:552 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:556 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:560 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:564 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:568 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:572 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:576 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:580 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:584 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
; SI-NEXT: s_cbranch_scc0 .LBB105_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v48
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v27
-; SI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v54
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v40
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v27, v59
-; SI-NEXT: v_lshrrev_b32_e32 v59, 16, v51
-; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v42
-; SI-NEXT: v_mov_b32_e32 v48, v32
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v43
-; SI-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v44
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v34
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v44
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v45
; SI-NEXT: s_mov_b64 s[4:5], 0
-; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v41
-; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v46
-; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v29
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v24
-; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v22
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v61
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v33
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v52
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v31
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v22
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v41
+; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v33
+; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v42
+; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v47
+; SI-NEXT: v_mov_b32_e32 v41, v60
+; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v45
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v57
-; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v12
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v28
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v56
-; SI-NEXT: v_mov_b32_e32 v56, v59
-; SI-NEXT: v_mov_b32_e32 v59, v27
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v35
-; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v59
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v37
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v43
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v39
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v4, v49
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v56
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v36
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v25
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v62
-; SI-NEXT: v_lshrrev_b32_e32 v62, 16, v54
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v20
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v58
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v55
-; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v47
-; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v14
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_mov_b32_e32 v32, v48
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v5
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v50
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v60
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v23
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v28
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v46
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v52
-; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v6
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v36
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v50
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v49
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v61
+; SI-NEXT: v_lshrrev_b32_e32 v61, 16, v38
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v34
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v25
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v53
+; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v20
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v26
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v59
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v38
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v53
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v29
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v39
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v62
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v31
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v32
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v48
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v26
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
-; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v14
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v27
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v18
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v30
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v18
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v35
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v30
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v37
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v4
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v10
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v24
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_mov_b32_e32 v0, v6
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v6
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v40
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v19
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v49
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v3
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v2
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v17
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v12
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(2)
+; SI-NEXT: v_mov_b32_e32 v2, v12
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v8
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v18, v0
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v13
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v15
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v9
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v8
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v51
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v10
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v11
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v5
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v19
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshrrev_b32_e32 v59, 16, v9
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v13
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v25
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v57
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v60
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
; SI-NEXT: s_branch .LBB105_3
; SI-NEXT: .LBB105_2:
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v2, v12
+; SI-NEXT: v_mov_b32_e32 v4, v49
+; SI-NEXT: v_mov_b32_e32 v0, v6
+; SI-NEXT: v_mov_b32_e32 v41, v60
; SI-NEXT: s_mov_b64 s[4:5], -1
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr52
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr41
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr3
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr55
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr51
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr46
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: ; implicit-def: $vgpr44
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr52
-; SI-NEXT: ; implicit-def: $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr22
-; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr31
+; SI-NEXT: ; implicit-def: $vgpr61
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr26
-; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr45
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr58
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr48
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr45
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr53
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr21
-; SI-NEXT: ; implicit-def: $vgpr19
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: ; implicit-def: $vgpr15
-; SI-NEXT: ; implicit-def: $vgpr13
-; SI-NEXT: ; implicit-def: $vgpr11
-; SI-NEXT: ; implicit-def: $vgpr9
; SI-NEXT: ; implicit-def: $vgpr7
-; SI-NEXT: ; implicit-def: $vgpr57
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr14
; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: ; implicit-def: $vgpr43
-; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr60
-; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: .LBB105_3: ; %Flow
+; SI-NEXT: v_mov_b32_e32 v5, v23
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:380 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
+; SI-NEXT: v_mov_b32_e32 v11, v56
+; SI-NEXT: v_mov_b32_e32 v6, v0
+; SI-NEXT: s_waitcnt expcnt(2)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:364 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:368 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:372 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:376 ; 4-byte Folded Reload
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
-; SI-NEXT: v_mov_b32_e32 v24, v53
; SI-NEXT: s_cmp_lg_u32 s4, 1
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_mov_b32_e32 v1, v0
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mov_b32_e32 v60, v56
; SI-NEXT: s_cbranch_scc1 .LBB105_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v20
-; SI-NEXT: v_add_f32_e32 v49, 0x40c00000, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v23
-; SI-NEXT: v_add_f32_e32 v33, 0x40c00000, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v8
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v2
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v40
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v4
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_lshrrev_b32_e32 v62, 16, v1
+; SI-NEXT: v_mov_b32_e32 v12, v2
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v50
; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v16
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_add_f32_e32 v61, 0x40c00000, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v59
-; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v14
-; SI-NEXT: v_add_f32_e32 v42, 0x40c00000, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v35
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v1
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v20
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_mov_b32_e32 v56, v1
+; SI-NEXT: v_mov_b32_e32 v55, v0
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v11
+; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v33
+; SI-NEXT: v_add_f32_e32 v52, 0x40c00000, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v10
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v1
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v12
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v4
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_mov_b32_e32 v11, v1
+; SI-NEXT: v_mov_b32_e32 v10, v0
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v51
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v6
+; SI-NEXT: v_add_f32_e32 v48, 0x40c00000, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v8
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_add_f32_e32 v26, 0x40c00000, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v46
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v32
+; SI-NEXT: v_add_f32_e32 v36, 0x40c00000, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v5
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v1
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v2
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v30
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v25
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v0
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v1
+; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v2
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v32
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v25
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v24
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v12
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_add_f32_e32 v52, 0x40c00000, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v10
-; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v54
-; SI-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:512 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v39
-; SI-NEXT: v_add_f32_e32 v26, 0x40c00000, v2
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
-; SI-NEXT: v_lshr_b64 v[59:60], v[33:34], 16
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: v_add_f32_e32 v45, 0x40c00000, v8
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v9, 0x40c00000, v9
-; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: v_add_f32_e32 v21, 0x40c00000, v8
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v9, 0x40c00000, v9
-; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:576 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v23
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v62, 16, v5
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v9
+; SI-NEXT: v_add_f32_e32 v61, 0x40c00000, v4
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v19
+; SI-NEXT: v_add_f32_e32 v59, 0x40c00000, v4
+; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v13
+; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v18
+; SI-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
+; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v2
+; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v3
+; SI-NEXT: v_lshr_b64 v[30:31], v[52:53], 16
+; SI-NEXT: v_mov_b32_e32 v32, v11
+; SI-NEXT: v_mov_b32_e32 v31, v10
+; SI-NEXT: v_lshr_b64 v[34:35], v[31:32], 16
+; SI-NEXT: v_mov_b32_e32 v50, v55
+; SI-NEXT: v_mov_b32_e32 v51, v56
+; SI-NEXT: v_lshr_b64 v[55:56], v[50:51], 16
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v51, 0x40c00000, v9
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v51
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:584 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v5
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v23, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v23
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:568 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v57, 0x40c00000, v4
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v41
+; SI-NEXT: v_add_f32_e32 v46, 0x40c00000, v4
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v24, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v24
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:560 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v5
+; SI-NEXT: v_mov_b32_e32 v12, v58
+; SI-NEXT: v_mov_b32_e32 v11, v57
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v25, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v25
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:556 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v5
+; SI-NEXT: v_mov_b32_e32 v10, v47
+; SI-NEXT: v_mov_b32_e32 v9, v46
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:552 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v44, 0x40c00000, v4
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:548 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v5
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:544 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v42, 0x40c00000, v4
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:540 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v5
+; SI-NEXT: v_mov_b32_e32 v23, v43
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:536 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v4
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:528 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v5
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:532 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v0
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:364 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:368 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:416 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:520 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:524 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v0
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:372 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:376 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v15, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v15
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v40, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v40
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:408 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:504 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v24, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v24
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:412 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:580 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v13, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v13
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v13, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v17, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v17
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:572 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v17
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v11, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v11
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:404 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:488 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v38, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v38
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v57, 0xffff0000, v38
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:484 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v0
+; SI-NEXT: v_add_f32_e32 v39, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v39
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_lshr_b64 v[0:1], v[6:7], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v10, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:384 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:464 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v0
+; SI-NEXT: v_lshr_b64 v[4:5], v[6:7], 16
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:380 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:384 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:468 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:456 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v25, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v25
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v3, 0xffff0000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:500 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v3, 0x40c00000, v3
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v3
+; SI-NEXT: v_lshr_b64 v[4:5], v[6:7], 16
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v35, 0xffff0000, v25
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v12, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v12
-; SI-NEXT: v_lshr_b64 v[54:55], v[8:9], 16
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:376 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:380 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:460 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:448 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v28, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v28
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[4:5], v[6:7], 16
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:396 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:400 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:496 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v2, 0x40c00000, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xffff0000, v30
-; SI-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:452 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:564 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v16, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v16
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[4:5], v[6:7], 16
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:332 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:336 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v41, 0xffff0000, v16
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v14, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v14
-; SI-NEXT: v_lshr_b64 v[29:30], v[8:9], 16
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:368 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:372 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:444 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:436 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v18, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v18
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[4:5], v[6:7], 16
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:388 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:392 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v5, v60
+; SI-NEXT: v_mov_b32_e32 v4, v59
+; SI-NEXT: v_lshr_b64 v[58:59], v[61:62], 16
+; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v18
+; SI-NEXT: v_mov_b32_e32 v59, v58
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v0
-; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v5
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:492 ; 4-byte Folded Reload
-; SI-NEXT: v_add_f32_e32 v0, 0x40c00000, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v18
-; SI-NEXT: v_add_f32_e32 v56, 0x40c00000, v6
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:476 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v55, v2
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v44, v29
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v16, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v16
-; SI-NEXT: v_lshr_b64 v[39:40], v[8:9], 16
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:360 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:364 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v5, 0xffff0000, v5
-; SI-NEXT: v_add_f32_e32 v5, 0x40c00000, v5
-; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v5
-; SI-NEXT: v_and_b32_e32 v32, 0xffff0000, v16
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:440 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v37, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v37
-; SI-NEXT: v_lshr_b64 v[47:48], v[8:9], 16
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:352 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:356 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:428 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v20, 0x40c00000, v7
+; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v20
+; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v33, 0xffff0000, v20
+; SI-NEXT: v_lshr_b64 v[19:20], v[48:49], 16
+; SI-NEXT: v_mov_b32_e32 v20, v2
+; SI-NEXT: s_waitcnt vmcnt(2) expcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:432 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:480 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v17, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v17
-; SI-NEXT: v_lshr_b64 v[35:36], v[8:9], 16
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:344 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:348 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:420 ; 4-byte Folded Reload
+; SI-NEXT: v_add_f32_e32 v21, 0x40c00000, v8
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v21
+; SI-NEXT: v_lshr_b64 v[6:7], v[7:8], 16
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v29, 0xffff0000, v21
+; SI-NEXT: v_mov_b32_e32 v21, v3
+; SI-NEXT: v_mov_b32_e32 v3, v15
+; SI-NEXT: v_mov_b32_e32 v2, v14
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(4) expcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:424 ; 4-byte Folded Reload
; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:472 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v41, 0xffff0000, v17
-; SI-NEXT: v_mov_b32_e32 v17, v56
-; SI-NEXT: v_and_b32_e32 v36, 0xffff0000, v14
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v18, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v18
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v1
+; SI-NEXT: v_mov_b32_e32 v1, v27
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: v_add_f32_e32 v22, 0x40c00000, v8
+; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v22
+; SI-NEXT: v_lshr_b64 v[6:7], v[7:8], 16
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(2)
+; SI-NEXT: v_mov_b32_e32 v1, v37
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:340 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v6
-; SI-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
-; SI-NEXT: v_and_b32_e32 v60, 0xffff0000, v18
-; SI-NEXT: v_mov_b32_e32 v18, v57
-; SI-NEXT: v_mov_b32_e32 v56, v3
-; SI-NEXT: v_mov_b32_e32 v3, v27
-; SI-NEXT: v_mov_b32_e32 v2, v26
-; SI-NEXT: v_lshr_b64 v[27:28], v[30:31], 16
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v26, 0xffff0000, v37
-; SI-NEXT: v_lshr_b64 v[37:38], v[52:53], 16
+; SI-NEXT: v_mov_b32_e32 v7, v49
+; SI-NEXT: v_and_b32_e32 v54, 0xffff0000, v22
+; SI-NEXT: v_mov_b32_e32 v22, v42
+; SI-NEXT: v_lshr_b64 v[42:43], v[26:27], 16
+; SI-NEXT: v_and_b32_e32 v26, 0xffff0000, v28
+; SI-NEXT: v_lshr_b64 v[27:28], v[36:37], 16
+; SI-NEXT: v_lshr_b64 v[36:37], v[14:15], 16
+; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v39
+; SI-NEXT: v_lshr_b64 v[38:39], v[11:12], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(2)
+; SI-NEXT: v_mov_b32_e32 v1, v3
+; SI-NEXT: v_lshr_b64 v[15:16], v[2:3], 16
+; SI-NEXT: v_lshr_b64 v[48:49], v[4:5], 16
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v37, 0xffff0000, v13
+; SI-NEXT: v_mov_b32_e32 v39, v38
+; SI-NEXT: v_mov_b32_e32 v49, v48
+; SI-NEXT: v_mov_b32_e32 v17, v15
; SI-NEXT: v_mov_b32_e32 v28, v27
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v19, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v19
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v0
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:332 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
-; SI-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_and_b32_e32 v58, 0xffff0000, v19
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v8
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff0000, v9
-; SI-NEXT: v_add_f32_e32 v20, 0x40c00000, v9
-; SI-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v20
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v8, v50
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v8, v34
-; SI-NEXT: v_and_b32_e32 v48, 0xffff0000, v20
-; SI-NEXT: v_mov_b32_e32 v9, v3
-; SI-NEXT: v_lshr_b64 v[49:50], v[49:50], 16
-; SI-NEXT: v_lshr_b64 v[33:34], v[30:31], 16
-; SI-NEXT: v_mov_b32_e32 v50, v39
-; SI-NEXT: v_lshr_b64 v[29:30], v[26:27], 16
-; SI-NEXT: v_mov_b32_e32 v34, v33
-; SI-NEXT: v_mov_b32_e32 v26, v35
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v8, v62
-; SI-NEXT: v_lshr_b64 v[61:62], v[61:62], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v8, v43
-; SI-NEXT: v_lshr_b64 v[42:43], v[42:43], 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: v_lshr_b64 v[38:39], v[41:42], 16
-; SI-NEXT: v_lshr_b64 v[39:40], v[60:61], 16
-; SI-NEXT: v_mov_b32_e32 v60, v59
-; SI-NEXT: v_lshr_b64 v[58:59], v[58:59], 16
-; SI-NEXT: v_mov_b32_e32 v43, v42
-; SI-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v20, v7
-; SI-NEXT: v_mov_b32_e32 v19, v6
-; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:508 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:512 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v6, 0xffff0000, v12
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_mov_b32_e32 v62, v8
+; SI-NEXT: v_mov_b32_e32 v1, v62
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v1, v5
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v10
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v8, 0xffff0000, v24
+; SI-NEXT: v_mov_b32_e32 v24, v42
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[9:10], v[2:3], 16
-; SI-NEXT: v_mov_b32_e32 v2, v56
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v12
+; SI-NEXT: v_lshr_b64 v[11:12], v[44:45], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v2, v5
-; SI-NEXT: v_and_b32_e32 v10, 0xffff0000, v11
-; SI-NEXT: v_lshr_b64 v[11:12], v[55:56], 16
-; SI-NEXT: v_and_b32_e32 v12, 0xffff0000, v13
-; SI-NEXT: v_lshr_b64 v[13:14], v[4:5], 16
-; SI-NEXT: v_and_b32_e32 v14, 0xffff0000, v15
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v55, v1
-; SI-NEXT: v_lshr_b64 v[15:16], v[0:1], 16
+; SI-NEXT: v_mov_b32_e32 v1, v10
+; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
+; SI-NEXT: v_and_b32_e32 v10, 0xffff0000, v40
+; SI-NEXT: v_mov_b32_e32 v40, v32
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v1, v45
+; SI-NEXT: v_lshr_b64 v[44:45], v[20:21], 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v1, v18
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v20
-; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_and_b32_e32 v20, 0xffff0000, v23
-; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v24
-; SI-NEXT: v_and_b32_e32 v16, 0xffff0000, v25
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v21
+; SI-NEXT: v_lshr_b64 v[21:22], v[22:23], 16
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v43, 0xffff0000, v0
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff0000, v51
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, v22
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, v23
+; SI-NEXT: v_lshr_b64 v[22:23], v[10:11], 16
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[21:22], 16
-; SI-NEXT: v_mov_b32_e32 v21, v46
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v21, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[21:22], v[45:46], 16
-; SI-NEXT: v_mov_b32_e32 v45, v1
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:404 ; 4-byte Folded Reload
-; SI-NEXT: v_lshr_b64 v[22:23], v[32:33], 16
-; SI-NEXT: v_mov_b32_e32 v46, v54
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_mov_b32_e32 v52, v53
-; SI-NEXT: v_lshr_b64 v[56:57], v[0:1], 16
-; SI-NEXT: v_mov_b32_e32 v57, v37
-; SI-NEXT: v_lshr_b64 v[0:1], v[20:21], 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[22:23], v[8:9], 16
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[18:19], 16
-; SI-NEXT: v_lshr_b64 v[1:2], v[16:17], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[22:23], v[37:38], 16
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[14:15], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[22:23], v[47:48], 16
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[12:13], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[22:23], v[57:58], 16
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[10:11], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[22:23], v[35:36], 16
+; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[8:9], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[22:23], v[41:42], 16
+; SI-NEXT: v_mov_b32_e32 v35, v34
+; SI-NEXT: v_mov_b32_e32 v37, v19
+; SI-NEXT: v_mov_b32_e32 v42, v55
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_and_b32_e32 v20, 0xffff0000, v0
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[6:7], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[43:44], 16
+; SI-NEXT: buffer_load_dword v45, off, s[0:3], s32 offset:396 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v46, off, s[0:3], s32 offset:400 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:380 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:384 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:372 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:376 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:364 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:368 ; 4-byte Folded Reload
+; SI-NEXT: v_lshr_b64 v[60:61], v[20:21], 16
+; SI-NEXT: v_lshr_b64 v[61:62], v[6:7], 16
+; SI-NEXT: v_mov_b32_e32 v43, v30
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_mov_b32_e32 v31, v2
+; SI-NEXT: v_mov_b32_e32 v2, v11
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mov_b32_e32 v52, v4
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:388 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:392 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:352 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:356 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:360 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[10:11], v[14:15], 16
+; SI-NEXT: v_lshr_b64 v[11:12], v[26:27], 16
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshr_b64 v[11:12], v[18:19], 16
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v2, v9
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[1:2], v[36:37], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v36, v49
-; SI-NEXT: v_lshr_b64 v[48:49], v[48:49], 16
-; SI-NEXT: v_mov_b32_e32 v41, v0
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_mov_b32_e32 v1, v61
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[11:12], v[33:34], 16
+; SI-NEXT: v_lshr_b64 v[8:9], v[29:30], 16
+; SI-NEXT: v_mov_b32_e32 v14, v36
+; SI-NEXT: v_mov_b32_e32 v36, v51
+; SI-NEXT: v_mov_b32_e32 v29, v10
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:340 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:344 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:348 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(11)
+; SI-NEXT: v_mov_b32_e32 v34, v4
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_lshr_b64 v[4:5], v[54:55], 16
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: .LBB105_5: ; %end
-; SI-NEXT: s_waitcnt expcnt(2)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v56
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v52
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_or_b32_e32 v0, v0, v1
+; SI-NEXT: v_and_b32_e32 v23, 0xffff, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v45
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v44
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v21
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v19
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v17
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:352 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:356 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:360 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v15, 0xffff, v17
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v7
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v15
-; SI-NEXT: v_and_b32_e32 v15, 0xffff, v9
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v13
-; SI-NEXT: v_and_b32_e32 v13, 0xffff, v11
-; SI-NEXT: v_and_b32_e32 v19, 0xffff, v57
+; SI-NEXT: v_and_b32_e32 v12, 0xffff, v49
+; SI-NEXT: v_and_b32_e32 v19, 0xffff, v14
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v59
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v61
+; SI-NEXT: v_and_b32_e32 v21, 0xffff, v28
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
-; SI-NEXT: v_and_b32_e32 v21, 0xffff, v34
-; SI-NEXT: v_and_b32_e32 v23, 0xffff, v28
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: v_readlane_b32 s30, v63, 3
; SI-NEXT: v_readlane_b32 s31, v63, 4
; SI-NEXT: v_readlane_b32 s36, v63, 2
; SI-NEXT: v_readlane_b32 s35, v63, 1
; SI-NEXT: v_readlane_b32 s34, v63, 0
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v25
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v27
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v3
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v60
; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v9
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:340 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:344 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:348 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v25
+; SI-NEXT: v_and_b32_e32 v10, 0xffff, v39
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v27
; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v41
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v9
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v25
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_or_b32_e32 v6, v6, v7
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v25
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_or_b32_e32 v8, v8, v9
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v55
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v25
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v9, v10, v9
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
; SI-NEXT: v_or_b32_e32 v10, v10, v11
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v25
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v11, v12, v11
-; SI-NEXT: v_and_b32_e32 v12, 0xffff, v51
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v12, 0xffff, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v12, v12, v14
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v25
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v13, v13, v14
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:176 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v14, v14, v16
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v25
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v15, v15, v16
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v46
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v16, 0xffff, v31
; SI-NEXT: v_or_b32_e32 v16, v16, v18
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v62
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v17, v17, v18
-; SI-NEXT: v_and_b32_e32 v18, 0xffff, v44
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v18, 0xffff, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v18, v18, v20
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v52
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v25
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v19, v19, v20
-; SI-NEXT: v_and_b32_e32 v20, 0xffff, v50
-; SI-NEXT: v_or_b32_e32 v20, v20, v22
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v31
-; SI-NEXT: v_or_b32_e32 v21, v21, v22
-; SI-NEXT: v_and_b32_e32 v22, 0xffff, v47
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v29
-; SI-NEXT: v_or_b32_e32 v22, v22, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_and_b32_e32 v20, 0xffff, v45
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_or_b32_e32 v20, v20, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
+; SI-NEXT: v_or_b32_e32 v21, v21, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:332 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: v_or_b32_e32 v22, v24, v22
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v23, v23, v24
-; SI-NEXT: v_and_b32_e32 v24, 0xffff, v26
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v38
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v34
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_or_b32_e32 v24, v24, v25
-; SI-NEXT: v_and_b32_e32 v25, 0xffff, v43
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v31
-; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v48
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v37
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
; SI-NEXT: v_or_b32_e32 v25, v25, v26
-; SI-NEXT: v_and_b32_e32 v26, 0xffff, v28
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v28
-; SI-NEXT: v_or_b32_e32 v26, v26, v27
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v27, 0xffff, v28
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v40
+; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_or_b32_e32 v26, v26, v27
+; SI-NEXT: v_and_b32_e32 v27, 0xffff, v35
; SI-NEXT: v_or_b32_e32 v27, v27, v28
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v58
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v53
; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_or_b32_e32 v28, v28, v29
-; SI-NEXT: v_and_b32_e32 v29, 0xffff, v60
+; SI-NEXT: v_and_b32_e32 v29, 0xffff, v43
+; SI-NEXT: v_or_b32_e32 v29, v29, v30
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v36
+; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; SI-NEXT: v_or_b32_e32 v30, v30, v31
+; SI-NEXT: v_and_b32_e32 v31, 0xffff, v42
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
@@ -223484,17 +223723,9 @@ define inreg <64 x i16> @bitcast_v64bf16_to_v64i16_scalar(<64 x bfloat> inreg %a
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:52 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v29, v29, v30
-; SI-NEXT: v_and_b32_e32 v30, 0xffff, v32
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v30, v30, v31
-; SI-NEXT: v_and_b32_e32 v31, 0xffff, v36
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v33
; SI-NEXT: v_or_b32_e32 v31, v31, v32
; SI-NEXT: s_or_saveexec_b64 s[4:5], -1
-; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:516 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:588 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -229238,36 +229469,34 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f32_f16_e32 v29, s76
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v0
-; SI-NEXT: v_or_b32_e32 v53, v1, v26
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s35
+; SI-NEXT: v_or_b32_e32 v35, v1, v32
; SI-NEXT: v_cvt_f16_f32_e32 v1, v2
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v3
-; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s20
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
-; SI-NEXT: v_or_b32_e32 v42, v2, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s22
-; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v0
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s20
+; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s34
+; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
-; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
-; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
+; SI-NEXT: v_or_b32_e32 v40, v2, v54
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s22
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
-; SI-NEXT: v_or_b32_e32 v1, v1, v51
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_or_b32_e32 v34, v1, v5
; SI-NEXT: v_cvt_f32_f16_e32 v1, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s63
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v0
+; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
+; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s24
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
-; SI-NEXT: v_or_b32_e32 v62, v2, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_or_b32_e32 v62, v2, v10
; SI-NEXT: v_cvt_f32_f16_e32 v2, vcc_hi
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -229279,13 +229508,13 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v2
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v2
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
-; SI-NEXT: v_or_b32_e32 v58, v1, v33
+; SI-NEXT: v_or_b32_e32 v58, v1, v12
; SI-NEXT: v_cvt_f32_f16_e32 v1, s95
; SI-NEXT: v_cvt_f32_f16_e32 v2, s28
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v0
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s31
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
@@ -229293,11 +229522,11 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
-; SI-NEXT: v_lshlrev_b32_e32 v40, 16, v1
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, vcc_lo
-; SI-NEXT: v_or_b32_e32 v56, v2, v11
+; SI-NEXT: v_or_b32_e32 v56, v2, v41
; SI-NEXT: v_cvt_f32_f16_e32 v2, s94
-; SI-NEXT: v_or_b32_e32 v19, v0, v40
+; SI-NEXT: v_or_b32_e32 v46, v0, v21
; SI-NEXT: v_cvt_f32_f16_e32 v0, s91
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v48, v1
@@ -229312,23 +229541,24 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s92
; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s79
-; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v1
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s77
; SI-NEXT: v_add_f32_e32 v49, 0x38000000, v2
; SI-NEXT: v_cvt_f32_f16_e32 v2, s74
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s63
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v0
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v0
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v3
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v1
+; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s61
; SI-NEXT: v_cvt_f16_f32_e32 v28, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s88
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v2
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v2
; SI-NEXT: v_cvt_f32_f16_e32 v2, s78
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
@@ -229342,16 +229572,16 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
-; SI-NEXT: v_cvt_f16_f32_e32 v35, v2
+; SI-NEXT: v_cvt_f16_f32_e32 v4, v2
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v22, v0, v1
+; SI-NEXT: v_or_b32_e32 v27, v0, v1
; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v35
-; SI-NEXT: v_or_b32_e32 v16, v2, v1
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v4
+; SI-NEXT: v_or_b32_e32 v52, v2, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s56
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f32_f16_e32 v2, s72
@@ -229359,75 +229589,73 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v2
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f16_f32_e32 v4, v1
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v3
; SI-NEXT: v_cvt_f32_f16_e32 v2, s47
-; SI-NEXT: v_or_b32_e32 v18, v0, v1
+; SI-NEXT: v_or_b32_e32 v16, v0, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s62
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v3, s46
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v4
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f16_f32_e32 v4, v1
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
-; SI-NEXT: v_or_b32_e32 v55, v2, v0
+; SI-NEXT: v_or_b32_e32 v18, v2, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s60
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v4
; SI-NEXT: v_or_b32_e32 v39, v1, v2
; SI-NEXT: v_cvt_f32_f16_e32 v1, s45
-; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f32_f16_e32 v2, s58
-; SI-NEXT: v_cvt_f16_f32_e32 v3, v0
+; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v19, v0
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v2
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f16_f32_e32 v4, v1
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v3
+; SI-NEXT: v_cvt_f16_f32_e32 v3, v1
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v19
; SI-NEXT: v_cvt_f32_f16_e32 v2, s44
; SI-NEXT: v_or_b32_e32 v25, v0, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s43
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v3
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v3, s29
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v4
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f16_f32_e32 v4, v1
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
-; SI-NEXT: v_or_b32_e32 v41, v2, v0
+; SI-NEXT: v_or_b32_e32 v22, v2, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s42
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v4
-; SI-NEXT: v_or_b32_e32 v12, v1, v2
+; SI-NEXT: v_or_b32_e32 v42, v1, v2
; SI-NEXT: v_cvt_f32_f16_e32 v1, s27
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f32_f16_e32 v2, s41
-; SI-NEXT: v_cvt_f16_f32_e32 v3, v0
+; SI-NEXT: v_cvt_f16_f32_e32 v45, v0
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v59, v1
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v3
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v45
; SI-NEXT: v_cvt_f32_f16_e32 v2, s25
-; SI-NEXT: v_or_b32_e32 v34, v0, v1
+; SI-NEXT: v_or_b32_e32 v13, v0, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s40
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v3, s23
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
-; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
+; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
; SI-NEXT: v_cvt_f16_f32_e32 v44, v1
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
@@ -229435,7 +229663,7 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v9, v2, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s15
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v44
-; SI-NEXT: v_or_b32_e32 v14, v1, v2
+; SI-NEXT: v_or_b32_e32 v11, v1, v2
; SI-NEXT: v_cvt_f32_f16_e32 v1, s21
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f32_f16_e32 v2, s14
@@ -229446,7 +229674,7 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v57, v1
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v47
; SI-NEXT: v_cvt_f32_f16_e32 v2, s19
-; SI-NEXT: v_or_b32_e32 v52, v0, v1
+; SI-NEXT: v_or_b32_e32 v6, v0, v1
; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
@@ -229456,10 +229684,10 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v57
-; SI-NEXT: v_or_b32_e32 v6, v2, v0
+; SI-NEXT: v_or_b32_e32 v55, v2, v0
; SI-NEXT: v_cvt_f32_f16_e32 v0, s11
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v60
-; SI-NEXT: v_or_b32_e32 v27, v1, v2
+; SI-NEXT: v_or_b32_e32 v33, v1, v2
; SI-NEXT: v_cvt_f32_f16_e32 v1, s10
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v3, v2
@@ -229480,10 +229708,10 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v50
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_or_b32_e32 v20, v1, v38
-; SI-NEXT: v_or_b32_e32 v49, v0, v17
+; SI-NEXT: v_or_b32_e32 v49, v0, v15
; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s8
-; SI-NEXT: v_or_b32_e32 v50, v2, v54
+; SI-NEXT: v_or_b32_e32 v50, v2, v17
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -229491,147 +229719,144 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
-; SI-NEXT: v_or_b32_e32 v0, v0, v15
+; SI-NEXT: v_or_b32_e32 v0, v0, v51
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: v_or_b32_e32 v37, v1, v21
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v37, v1, v26
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshr_b64 v[0:1], v[26:27], 16
+; SI-NEXT: v_lshr_b64 v[0:1], v[32:33], 16
; SI-NEXT: v_or_b32_e32 v36, v2, v28
-; SI-NEXT: v_lshr_b64 v[1:2], v[5:6], 16
-; SI-NEXT: v_mov_b32_e32 v2, v9
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: v_lshr_b64 v[1:2], v[54:55], 16
+; SI-NEXT: v_mov_b32_e32 v2, v13
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v2, v42
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v2, v34
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v4, v48, v24
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v4, s6
-; SI-NEXT: v_or_b32_e32 v32, v48, v24
-; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mov_b32_e32 v45, v27
+; SI-NEXT: v_mov_b32_e32 v53, v55
+; SI-NEXT: v_mov_b32_e32 v55, v6
+; SI-NEXT: v_mov_b32_e32 v23, v11
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
-; SI-NEXT: v_mov_b32_e32 v27, v6
-; SI-NEXT: v_lshr_b64 v[9:10], v[33:34], 16
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v2, v12
-; SI-NEXT: v_or_b32_e32 v43, v4, v30
-; SI-NEXT: v_lshr_b64 v[4:5], v[51:52], 16
-; SI-NEXT: v_lshr_b64 v[5:6], v[13:14], 16
-; SI-NEXT: v_lshr_b64 v[12:13], v[11:12], 16
-; SI-NEXT: v_mov_b32_e32 v11, v59
-; SI-NEXT: v_mov_b32_e32 v59, v20
-; SI-NEXT: v_mov_b32_e32 v20, v50
-; SI-NEXT: v_mov_b32_e32 v23, v52
-; SI-NEXT: v_mov_b32_e32 v51, v53
-; SI-NEXT: v_mov_b32_e32 v26, v39
+; SI-NEXT: v_mov_b32_e32 v7, v33
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v2, v41
-; SI-NEXT: v_lshr_b64 v[52:53], v[38:39], 16
-; SI-NEXT: v_lshr_b64 v[38:39], v[17:18], 16
-; SI-NEXT: v_mov_b32_e32 v46, v14
-; SI-NEXT: v_lshr_b64 v[13:14], v[40:41], 16
-; SI-NEXT: v_lshr_b64 v[53:54], v[54:55], 16
-; SI-NEXT: v_lshr_b64 v[39:40], v[15:16], 16
-; SI-NEXT: v_mov_b32_e32 v40, v37
-; SI-NEXT: v_mov_b32_e32 v37, v35
-; SI-NEXT: v_mov_b32_e32 v54, v36
-; SI-NEXT: v_lshr_b64 v[35:36], v[30:31], 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v2, v22
+; SI-NEXT: v_or_b32_e32 v43, v4, v30
+; SI-NEXT: v_lshr_b64 v[4:5], v[5:6], 16
+; SI-NEXT: v_lshr_b64 v[5:6], v[10:11], 16
+; SI-NEXT: v_mov_b32_e32 v11, v9
+; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
+; SI-NEXT: v_lshr_b64 v[9:10], v[12:13], 16
+; SI-NEXT: v_lshr_b64 v[12:13], v[41:42], 16
+; SI-NEXT: v_lshr_b64 v[13:14], v[21:22], 16
+; SI-NEXT: v_mov_b32_e32 v33, v40
+; SI-NEXT: v_mov_b32_e32 v22, v39
+; SI-NEXT: v_lshr_b64 v[40:41], v[38:39], 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v2, v25
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mov_b32_e32 v25, v44
-; SI-NEXT: v_mov_b32_e32 v44, v49
-; SI-NEXT: v_lshr_b64 v[48:49], v[21:22], 16
+; SI-NEXT: v_lshr_b64 v[38:39], v[17:18], 16
+; SI-NEXT: v_mov_b32_e32 v25, v49
+; SI-NEXT: v_lshr_b64 v[48:49], v[26:27], 16
+; SI-NEXT: v_mov_b32_e32 v39, v50
; SI-NEXT: v_lshr_b64 v[49:50], v[28:29], 16
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
-; SI-NEXT: v_mov_b32_e32 v33, v42
-; SI-NEXT: v_mov_b32_e32 v42, v18
-; SI-NEXT: v_mov_b32_e32 v41, v16
-; SI-NEXT: v_mov_b32_e32 v6, v5
+; SI-NEXT: v_mov_b32_e32 v50, v36
+; SI-NEXT: v_mov_b32_e32 v32, v44
+; SI-NEXT: v_mov_b32_e32 v21, v45
+; SI-NEXT: v_lshr_b64 v[44:45], v[15:16], 16
+; SI-NEXT: v_mov_b32_e32 v45, v52
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v2, v55
+; SI-NEXT: v_mov_b32_e32 v2, v16
+; SI-NEXT: v_lshr_b64 v[41:42], v[51:52], 16
+; SI-NEXT: v_lshr_b64 v[51:52], v[30:31], 16
+; SI-NEXT: v_mov_b32_e32 v54, v18
+; SI-NEXT: v_mov_b32_e32 v6, v5
; SI-NEXT: v_mov_b32_e32 v10, v9
; SI-NEXT: v_mov_b32_e32 v14, v13
-; SI-NEXT: v_mov_b32_e32 v55, v22
; SI-NEXT: v_mov_b32_e32 v30, v3
-; SI-NEXT: v_mov_b32_e32 v36, v7
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v2, v1
+; SI-NEXT: v_mov_b32_e32 v1, v27
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_branch .LBB109_6
; SI-NEXT: .LBB109_5:
; SI-NEXT: v_mov_b32_e32 v0, s73
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s72
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s62
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s60
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s58
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s43
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s42
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s9
-; SI-NEXT: v_mov_b32_e32 v1, s25
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v0, vcc_lo
; SI-NEXT: v_mov_b32_e32 v1, s27
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v1, s29
; SI-NEXT: v_mov_b32_e32 v36, s88
; SI-NEXT: v_mov_b32_e32 v30, s76
; SI-NEXT: v_mov_b32_e32 v28, s78
-; SI-NEXT: v_mov_b32_e32 v37, s75
-; SI-NEXT: v_mov_b32_e32 v11, s41
-; SI-NEXT: v_mov_b32_e32 v25, s40
+; SI-NEXT: v_mov_b32_e32 v27, s75
+; SI-NEXT: v_mov_b32_e32 v19, s60
+; SI-NEXT: v_mov_b32_e32 v21, s42
+; SI-NEXT: v_mov_b32_e32 v59, s41
+; SI-NEXT: v_mov_b32_e32 v32, s40
; SI-NEXT: v_mov_b32_e32 v47, s15
; SI-NEXT: v_mov_b32_e32 v57, s14
; SI-NEXT: v_mov_b32_e32 v60, s13
; SI-NEXT: v_mov_b32_e32 v43, s6
-; SI-NEXT: v_mov_b32_e32 v54, s7
-; SI-NEXT: v_mov_b32_e32 v40, s8
-; SI-NEXT: v_mov_b32_e32 v44, s12
-; SI-NEXT: v_mov_b32_e32 v20, s90
-; SI-NEXT: v_mov_b32_e32 v59, s92
-; SI-NEXT: v_mov_b32_e32 v32, vcc_lo
-; SI-NEXT: v_mov_b32_e32 v19, s31
+; SI-NEXT: v_mov_b32_e32 v50, s7
+; SI-NEXT: v_mov_b32_e32 v37, s8
+; SI-NEXT: v_mov_b32_e32 v25, s12
+; SI-NEXT: v_mov_b32_e32 v39, s90
+; SI-NEXT: v_mov_b32_e32 v20, s92
+; SI-NEXT: v_mov_b32_e32 v46, s31
; SI-NEXT: v_mov_b32_e32 v56, s28
; SI-NEXT: v_mov_b32_e32 v58, s26
; SI-NEXT: v_mov_b32_e32 v61, s24
; SI-NEXT: v_mov_b32_e32 v62, s22
; SI-NEXT: v_mov_b32_e32 v34, s20
; SI-NEXT: v_mov_b32_e32 v33, s18
-; SI-NEXT: v_mov_b32_e32 v51, s16
-; SI-NEXT: v_mov_b32_e32 v45, s17
-; SI-NEXT: v_mov_b32_e32 v27, s19
-; SI-NEXT: v_mov_b32_e32 v23, s21
-; SI-NEXT: v_mov_b32_e32 v46, s23
-; SI-NEXT: v_mov_b32_e32 v26, s46
-; SI-NEXT: v_mov_b32_e32 v42, s56
-; SI-NEXT: v_mov_b32_e32 v41, s57
-; SI-NEXT: v_mov_b32_e32 v55, s59
+; SI-NEXT: v_mov_b32_e32 v35, s16
+; SI-NEXT: v_mov_b32_e32 v7, s17
+; SI-NEXT: v_mov_b32_e32 v53, s19
+; SI-NEXT: v_mov_b32_e32 v55, s21
+; SI-NEXT: v_mov_b32_e32 v23, s23
+; SI-NEXT: v_mov_b32_e32 v11, s25
+; SI-NEXT: v_mov_b32_e32 v22, s46
+; SI-NEXT: v_mov_b32_e32 v54, s47
+; SI-NEXT: v_mov_b32_e32 v45, s57
; SI-NEXT: v_mov_b32_e32 v29, s11
; SI-NEXT: v_mov_b32_e32 v31, s10
; SI-NEXT: v_mov_b32_e32 v2, s36
@@ -229642,81 +229867,84 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v12, s93
; SI-NEXT: v_mov_b32_e32 v14, s95
; SI-NEXT: v_mov_b32_e32 v24, s94
-; SI-NEXT: v_mov_b32_e32 v52, s91
-; SI-NEXT: v_mov_b32_e32 v53, s89
-; SI-NEXT: v_mov_b32_e32 v38, s79
-; SI-NEXT: v_mov_b32_e32 v39, s77
+; SI-NEXT: v_mov_b32_e32 v40, s91
+; SI-NEXT: v_mov_b32_e32 v38, s89
+; SI-NEXT: v_mov_b32_e32 v44, s79
+; SI-NEXT: v_mov_b32_e32 v41, s77
; SI-NEXT: v_mov_b32_e32 v48, s74
; SI-NEXT: v_mov_b32_e32 v49, s63
-; SI-NEXT: v_mov_b32_e32 v35, s61
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s29
+; SI-NEXT: v_mov_b32_e32 v51, s61
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v1, s44
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v1, s45
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s47
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v1, s56
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v1, s59
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_mov_b32_e32 v0, s37
; SI-NEXT: .LBB109_6: ; %end
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v51
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v35
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v45
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v60
; SI-NEXT: v_or_b32_e32 v1, v1, v3
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v33
; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v27
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v53
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v57
; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: s_waitcnt vmcnt(13)
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v34
; SI-NEXT: v_or_b32_e32 v4, v5, v4
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v23
+; SI-NEXT: v_and_b32_e32 v5, 0xffff, v55
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v47
; SI-NEXT: v_or_b32_e32 v5, v5, v9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v62
; SI-NEXT: v_or_b32_e32 v6, v9, v6
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v46
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v25
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v23
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v32
; SI-NEXT: v_or_b32_e32 v7, v7, v9
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v61
; SI-NEXT: v_or_b32_e32 v8, v9, v8
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v11
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v11
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v59
+; SI-NEXT: v_or_b32_e32 v9, v9, v13
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v58
+; SI-NEXT: v_or_b32_e32 v10, v13, v10
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v21
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
-; SI-NEXT: v_and_b32_e32 v23, 0xffff, v42
-; SI-NEXT: v_and_b32_e32 v25, 0xffff, v41
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v44
; SI-NEXT: v_and_b32_e32 v29, 0xffff, v29
; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
; SI-NEXT: v_or_b32_e32 v29, v29, v30
; SI-NEXT: v_and_b32_e32 v30, 0xffff, v43
-; SI-NEXT: v_and_b32_e32 v32, 0xffff, v32
-; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: s_waitcnt vmcnt(8)
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v31, 0xffff, v31
; SI-NEXT: v_readlane_b32 s30, v63, 4
; SI-NEXT: v_readlane_b32 s31, v63, 5
@@ -229724,78 +229952,66 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s36, v63, 2
; SI-NEXT: v_readlane_b32 s35, v63, 1
; SI-NEXT: v_readlane_b32 s34, v63, 0
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v9, 0xffff, v16
-; SI-NEXT: v_or_b32_e32 v9, v9, v13
-; SI-NEXT: v_and_b32_e32 v13, 0xffff, v58
-; SI-NEXT: v_or_b32_e32 v10, v13, v10
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
-; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v15, 0xffff, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v17, 0xffff, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v11, v11, v13
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v56
; SI-NEXT: v_or_b32_e32 v12, v13, v12
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v34
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v32, 0xffff, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v17, 0xffff, v18
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
+; SI-NEXT: v_and_b32_e32 v19, 0xffff, v22
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v17, v17, v18
+; SI-NEXT: v_and_b32_e32 v18, 0xffff, v20
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v38
; SI-NEXT: v_or_b32_e32 v13, v13, v33
-; SI-NEXT: v_and_b32_e32 v33, 0xffff, v19
-; SI-NEXT: v_and_b32_e32 v19, 0xffff, v26
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v38
+; SI-NEXT: v_and_b32_e32 v33, 0xffff, v46
; SI-NEXT: v_or_b32_e32 v14, v33, v14
-; SI-NEXT: v_or_b32_e32 v19, v19, v21
-; SI-NEXT: v_and_b32_e32 v21, 0xffff, v20
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v53
-; SI-NEXT: v_or_b32_e32 v20, v21, v20
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v21, 0xffff, v22
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v15, 0xffff, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v16, v32, v16
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v52
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v40
+; SI-NEXT: v_or_b32_e32 v18, v18, v32
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v49
+; SI-NEXT: v_and_b32_e32 v23, 0xffff, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
-; SI-NEXT: v_or_b32_e32 v21, v21, v22
-; SI-NEXT: v_and_b32_e32 v22, 0xffff, v44
-; SI-NEXT: v_or_b32_e32 v22, v22, v26
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_or_b32_e32 v23, v23, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v39
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
-; SI-NEXT: v_or_b32_e32 v17, v17, v18
-; SI-NEXT: v_and_b32_e32 v18, 0xffff, v59
-; SI-NEXT: v_or_b32_e32 v18, v18, v32
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v49
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
-; SI-NEXT: v_or_b32_e32 v24, v24, v26
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v37
-; SI-NEXT: v_or_b32_e32 v25, v25, v26
-; SI-NEXT: v_and_b32_e32 v26, 0xffff, v40
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_or_b32_e32 v19, v19, v21
+; SI-NEXT: v_and_b32_e32 v21, 0xffff, v39
+; SI-NEXT: v_or_b32_e32 v20, v21, v20
+; SI-NEXT: v_and_b32_e32 v21, 0xffff, v54
+; SI-NEXT: v_or_b32_e32 v21, v21, v22
+; SI-NEXT: v_and_b32_e32 v22, 0xffff, v25
+; SI-NEXT: v_or_b32_e32 v22, v22, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v41
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v45
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
@@ -229811,12 +230027,19 @@ define inreg <64 x i16> @bitcast_v64f16_to_v64i16_scalar(<64 x half> inreg %a, i
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:52 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: v_or_b32_e32 v24, v24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
+; SI-NEXT: v_or_b32_e32 v25, v25, v26
+; SI-NEXT: v_and_b32_e32 v26, 0xffff, v37
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v48
; SI-NEXT: v_or_b32_e32 v26, v26, v27
-; SI-NEXT: v_and_b32_e32 v27, 0xffff, v55
+; SI-NEXT: v_and_b32_e32 v27, 0xffff, v33
; SI-NEXT: v_or_b32_e32 v27, v27, v28
-; SI-NEXT: v_and_b32_e32 v28, 0xffff, v54
+; SI-NEXT: v_and_b32_e32 v28, 0xffff, v50
; SI-NEXT: v_or_b32_e32 v28, v28, v32
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v51
; SI-NEXT: v_or_b32_e32 v30, v30, v32
; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v36
; SI-NEXT: v_or_b32_e32 v31, v31, v32
@@ -230372,411 +230595,400 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:16 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:12 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:4
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v0
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v2
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v4
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v50
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v6
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v49
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v4
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v48
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v10
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v51
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v39
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v50
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v8
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v38
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v49
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v28
+; SI-NEXT: v_lshrrev_b32_e32 v59, 16, v10
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v58
-; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v1
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v24
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
-; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v3
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v58
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v60
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v5
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v48
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v63
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v7
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v59
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v58
+; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v1
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v30
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v26
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v61, 16, v3
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v60
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v9
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v58
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v57
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v5
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v40
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v56, 16, v11
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v61
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v47, 16, v7
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v54
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v13
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v57
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v9
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:328 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v56
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v15
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v47
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v45, 16, v11
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v46
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v17
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v46
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v22
+; SI-NEXT: v_lshrrev_b32_e32 v44, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v12
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v45
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v30
-; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v45
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v29
; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v59, 16, v26
-; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v25
+; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v25
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v24
; SI-NEXT: v_lshrrev_b32_e32 v41, 16, v23
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v22
; SI-NEXT: v_lshrrev_b32_e32 v42, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v20
-; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v18
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v16
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v14
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v12
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v20
+; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v18
+; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v16
+; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v14
+; SI-NEXT: v_lshlrev_b32_e32 v59, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v35
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v52
-; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v33
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v32
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v44
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
-; SI-NEXT: v_lshlrev_b32_e32 v47, 16, v37
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v44
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v56
; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v36
-; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v31
-; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v59
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v43
+; SI-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v57
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v48, 16, v43
-; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v42
-; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v41
-; SI-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v52, 16, v52
+; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v51
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v55
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v53
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v52
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
+; SI-NEXT: v_lshlrev_b32_e32 v53, 16, v53
+; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v42
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v41
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v40
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v55
+; SI-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v54
+; SI-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
; SI-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr61
-; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: ; implicit-def: $vgpr59
-; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: ; implicit-def: $vgpr63
+; SI-NEXT: ; implicit-def: $vgpr60
; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: ; implicit-def: $vgpr45
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr41
+; SI-NEXT: ; implicit-def: $vgpr40
; SI-NEXT: ; implicit-def: $vgpr55
-; SI-NEXT: ; implicit-def: $vgpr53
-; SI-NEXT: ; kill: killed $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; kill: killed $vgpr8
+; SI-NEXT: ; kill: killed $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; kill: killed $vgpr6
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB110_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v58
-; SI-NEXT: v_or_b32_e32 v61, v1, v8
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v58
+; SI-NEXT: v_or_b32_e32 v62, v1, v6
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v63
-; SI-NEXT: v_or_b32_e32 v46, v1, v3
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v61
+; SI-NEXT: v_or_b32_e32 v63, v1, v3
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v60
-; SI-NEXT: v_or_b32_e32 v62, v1, v3
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v57
+; SI-NEXT: v_or_b32_e32 v60, v1, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v40
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v59, v1, v3
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v54
-; SI-NEXT: v_or_b32_e32 v57, v1, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_alignbit_b32 v1, v61, v7, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v56, v1, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v9
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v47, v1, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: v_alignbit_b32 v1, v62, v5, 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_or_b32_e32 v56, v1, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v1, v46, v8, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v46, v1, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: v_alignbit_b32 v1, v63, v6, 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v13
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_or_b32_e32 v45, v1, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v1, v62, v9, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v1, v60, v7, 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v44, v1, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v1, v59, v11, 16
+; SI-NEXT: v_or_b32_e32 v44, v1, v51
+; SI-NEXT: v_alignbit_b32 v1, v56, v9, 16
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v17
-; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v43, v1, v3
-; SI-NEXT: v_alignbit_b32 v1, v57, v13, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v43, v1, v52
+; SI-NEXT: v_alignbit_b32 v1, v47, v11, 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v19
-; SI-NEXT: v_or_b32_e32 v42, v1, v48
-; SI-NEXT: v_alignbit_b32 v1, v56, v15, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v42, v1, v53
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_alignbit_b32 v1, v46, v13, 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v21
-; SI-NEXT: v_or_b32_e32 v41, v1, v49
-; SI-NEXT: v_alignbit_b32 v1, v45, v34, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v41, v1, v50
+; SI-NEXT: v_alignbit_b32 v1, v45, v59, 16
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v23
-; SI-NEXT: v_or_b32_e32 v55, v1, v50
-; SI-NEXT: v_alignbit_b32 v1, v44, v47, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
+; SI-NEXT: v_or_b32_e32 v40, v1, v0
+; SI-NEXT: v_alignbit_b32 v0, v44, v36, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v25
-; SI-NEXT: v_or_b32_e32 v53, v1, v51
-; SI-NEXT: v_alignbit_b32 v1, v43, v35, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v25
+; SI-NEXT: v_or_b32_e32 v55, v0, v2
+; SI-NEXT: v_alignbit_b32 v0, v43, v31, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v27
-; SI-NEXT: v_or_b32_e32 v5, v1, v39
-; SI-NEXT: v_alignbit_b32 v1, v42, v32, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v27
+; SI-NEXT: v_or_b32_e32 v3, v0, v4
+; SI-NEXT: v_alignbit_b32 v0, v42, v37, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v29
-; SI-NEXT: v_or_b32_e32 v3, v1, v4
-; SI-NEXT: v_alignbit_b32 v1, v41, v36, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v29
+; SI-NEXT: v_or_b32_e32 v1, v0, v54
+; SI-NEXT: v_alignbit_b32 v0, v41, v38, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v33
-; SI-NEXT: v_or_b32_e32 v4, v1, v6
-; SI-NEXT: v_alignbit_b32 v1, v55, v37, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v32
+; SI-NEXT: v_or_b32_e32 v2, v0, v35
+; SI-NEXT: v_alignbit_b32 v0, v40, v39, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr15
+; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr48
-; SI-NEXT: ; implicit-def: $vgpr49
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr51
-; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr52
+; SI-NEXT: ; implicit-def: $vgpr53
+; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: ; implicit-def: $vgpr54
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v1, v1, v7
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_alignbit_b32 v1, v53, v38, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v0, v0, v5
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr7
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v1, v1, v8
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v1, v5, v31, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v55, v33, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; SI-NEXT: ; implicit-def: $vgpr5
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v1, v1, v9
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v0, v0, v6
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v1, v3, v0, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v3, v48, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; SI-NEXT: ; implicit-def: $vgpr3
-; SI-NEXT: ; implicit-def: $vgpr9
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v1, v1, v11
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v0, v0, v7
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v1, v4, v2, 16
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v1, v49, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr11
-; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: ; implicit-def: $vgpr7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v1, v1, v13
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v0, v0, v9
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_alignbit_b32 v0, v2, v34, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v10
-; SI-NEXT: v_or_b32_e32 v1, v1, v15
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v8
+; SI-NEXT: v_or_b32_e32 v0, v0, v11
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v12
-; SI-NEXT: v_or_b32_e32 v1, v1, v34
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v10
+; SI-NEXT: v_or_b32_e32 v0, v0, v13
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v14
-; SI-NEXT: v_or_b32_e32 v1, v1, v47
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v12
+; SI-NEXT: v_or_b32_e32 v0, v0, v59
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v16
-; SI-NEXT: v_or_b32_e32 v1, v1, v35
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v14
+; SI-NEXT: v_or_b32_e32 v0, v0, v36
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v18
-; SI-NEXT: v_or_b32_e32 v1, v1, v32
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v16
+; SI-NEXT: v_or_b32_e32 v0, v0, v31
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v20
-; SI-NEXT: v_or_b32_e32 v1, v1, v36
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v18
+; SI-NEXT: v_or_b32_e32 v0, v0, v37
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v22
-; SI-NEXT: v_or_b32_e32 v1, v1, v37
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v20
+; SI-NEXT: v_or_b32_e32 v0, v0, v38
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v24
-; SI-NEXT: v_or_b32_e32 v1, v1, v38
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v22
+; SI-NEXT: v_or_b32_e32 v0, v0, v39
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v26
-; SI-NEXT: v_or_b32_e32 v1, v1, v31
-; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v24
+; SI-NEXT: v_or_b32_e32 v0, v0, v33
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v28
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v26
+; SI-NEXT: v_or_b32_e32 v0, v0, v48
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v30
-; SI-NEXT: v_or_b32_e32 v0, v0, v2
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
+; SI-NEXT: v_or_b32_e32 v0, v0, v34
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
@@ -230810,12 +231022,13 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: ; implicit-def: $vgpr8
+; SI-NEXT: ; implicit-def: $vgpr9
; SI-NEXT: ; implicit-def: $vgpr10
+; SI-NEXT: ; implicit-def: $vgpr11
; SI-NEXT: ; implicit-def: $vgpr12
; SI-NEXT: ; implicit-def: $vgpr13
; SI-NEXT: ; implicit-def: $vgpr14
-; SI-NEXT: ; implicit-def: $vgpr15
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr20
@@ -230824,138 +231037,132 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr34
-; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr0
+; SI-NEXT: ; implicit-def: $vgpr39
; SI-NEXT: ; implicit-def: $vgpr2
+; SI-NEXT: ; implicit-def: $vgpr33
+; SI-NEXT: ; implicit-def: $vgpr48
+; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: .LBB110_2: ; %Flow
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB110_4
; SI-NEXT: ; %bb.3: ; %cmp.true
+; SI-NEXT: v_add_i32_e32 v32, vcc, 3, v32
; SI-NEXT: v_add_i32_e32 v28, vcc, 3, v28
+; SI-NEXT: v_add_i32_e32 v29, vcc, 3, v29
+; SI-NEXT: v_add_i32_e32 v26, vcc, 3, v26
+; SI-NEXT: v_and_b32_e32 v32, 0xffff, v32
; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
+; SI-NEXT: v_and_b32_e32 v29, 0xffff, v29
+; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
+; SI-NEXT: v_or_b32_e32 v32, v35, v32
+; SI-NEXT: v_or_b32_e32 v28, v49, v28
+; SI-NEXT: v_or_b32_e32 v29, v54, v29
+; SI-NEXT: v_or_b32_e32 v26, v48, v26
+; SI-NEXT: v_add_i32_e32 v6, vcc, 0x30000, v32
+; SI-NEXT: v_add_i32_e32 v32, vcc, 0x30000, v28
+; SI-NEXT: v_add_i32_e32 v28, vcc, 0x30000, v29
+; SI-NEXT: v_add_i32_e32 v29, vcc, 0x30000, v26
+; SI-NEXT: v_add_i32_e32 v26, vcc, 3, v27
+; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
+; SI-NEXT: v_or_b32_e32 v4, v4, v26
+; SI-NEXT: v_add_i32_e32 v26, vcc, 0x30000, v4
+; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v24
+; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
+; SI-NEXT: v_or_b32_e32 v4, v33, v4
+; SI-NEXT: v_add_i32_e32 v24, vcc, 0x30000, v4
+; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v25
+; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
+; SI-NEXT: v_or_b32_e32 v2, v2, v4
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v0, v28
-; SI-NEXT: v_add_i32_e32 v28, vcc, 3, v29
-; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
-; SI-NEXT: v_or_b32_e32 v4, v4, v28
-; SI-NEXT: v_add_i32_e32 v28, vcc, 0x30000, v0
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v26
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v31, v0
-; SI-NEXT: v_add_i32_e32 v30, vcc, 3, v30
-; SI-NEXT: v_add_i32_e32 v26, vcc, 0x30000, v0
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v27
-; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v2, v2, v30
-; SI-NEXT: v_or_b32_e32 v0, v39, v0
-; SI-NEXT: v_add_i32_e32 v8, vcc, 0x30000, v2
-; SI-NEXT: v_add_i32_e32 v2, vcc, 0x30000, v4
-; SI-NEXT: v_add_i32_e32 v4, vcc, 0x30000, v0
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v24
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v38, v0
-; SI-NEXT: v_add_i32_e32 v24, vcc, 0x30000, v0
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v25
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v51, v0
-; SI-NEXT: v_add_i32_e32 v53, vcc, 0x30000, v0
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v22
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v37, v0
-; SI-NEXT: v_add_i32_e32 v22, vcc, 0x30000, v0
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v23
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v50, v0
-; SI-NEXT: v_add_i32_e32 v55, vcc, 0x30000, v0
+; SI-NEXT: v_add_i32_e32 v55, vcc, 0x30000, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v22
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v39, v2
+; SI-NEXT: v_add_i32_e32 v4, vcc, 0x30000, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v23
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v0, v0, v2
+; SI-NEXT: v_add_i32_e32 v40, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v20
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v36, v0
-; SI-NEXT: v_add_i32_e32 v20, vcc, 0x30000, v0
+; SI-NEXT: v_or_b32_e32 v0, v38, v0
+; SI-NEXT: v_add_i32_e32 v2, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v21
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v49, v0
+; SI-NEXT: v_or_b32_e32 v0, v50, v0
; SI-NEXT: v_add_i32_e32 v41, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v18
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v32, v0
+; SI-NEXT: v_or_b32_e32 v0, v37, v0
; SI-NEXT: v_add_i32_e32 v18, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v19
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v48, v0
+; SI-NEXT: v_or_b32_e32 v0, v53, v0
; SI-NEXT: v_add_i32_e32 v42, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v16
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v35, v0
+; SI-NEXT: v_or_b32_e32 v0, v31, v0
; SI-NEXT: v_add_i32_e32 v16, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_add_i32_e32 v30, vcc, 3, v33
-; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
-; SI-NEXT: v_or_b32_e32 v6, v6, v30
-; SI-NEXT: v_add_i32_e32 v6, vcc, 0x30000, v6
-; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v6
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v17, v0
+; SI-NEXT: v_or_b32_e32 v0, v52, v0
; SI-NEXT: v_add_i32_e32 v43, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v14
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v47, v0
+; SI-NEXT: v_or_b32_e32 v0, v36, v0
; SI-NEXT: v_add_i32_e32 v14, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v15, v0
+; SI-NEXT: v_or_b32_e32 v0, v51, v0
; SI-NEXT: v_add_i32_e32 v44, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v12
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v0, v34, v0
+; SI-NEXT: v_or_b32_e32 v0, v59, v0
; SI-NEXT: v_add_i32_e32 v12, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v13
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_add_i32_e32 v30, vcc, 3, v30
+; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
+; SI-NEXT: v_or_b32_e32 v30, v34, v30
+; SI-NEXT: v_add_i32_e32 v30, vcc, 0x30000, v30
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v13, v0
; SI-NEXT: v_add_i32_e32 v45, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v10
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v10, v0
; SI-NEXT: v_add_i32_e32 v10, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v11, v0
-; SI-NEXT: v_add_i32_e32 v56, vcc, 0x30000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
+; SI-NEXT: v_add_i32_e32 v46, vcc, 0x30000, v0
+; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v0, v11, v0
-; SI-NEXT: v_add_i32_e32 v11, vcc, 0x30000, v0
+; SI-NEXT: v_or_b32_e32 v0, v8, v0
+; SI-NEXT: v_add_i32_e32 v8, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v9, v0
-; SI-NEXT: v_add_i32_e32 v57, vcc, 0x30000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v54, 16, v57
+; SI-NEXT: v_add_i32_e32 v47, vcc, 0x30000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
@@ -230963,14 +231170,13 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v0, v9, v0
; SI-NEXT: v_add_i32_e32 v9, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v7, v0
-; SI-NEXT: v_add_i32_e32 v59, vcc, 0x30000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v40, 16, v59
+; SI-NEXT: v_add_i32_e32 v56, vcc, 0x30000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
@@ -230978,14 +231184,14 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v0, v7, v0
; SI-NEXT: v_add_i32_e32 v7, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v5
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v5, v0
-; SI-NEXT: v_add_i32_e32 v62, vcc, 0x30000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v62
+; SI-NEXT: v_add_i32_e32 v60, vcc, 0x30000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v60
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
@@ -230993,14 +231199,14 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v0, v5, v0
; SI-NEXT: v_add_i32_e32 v5, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v3, v0
-; SI-NEXT: v_add_i32_e32 v46, vcc, 0x30000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v63, 16, v46
+; SI-NEXT: v_add_i32_e32 v63, vcc, 0x30000, v0
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v61, 16, v63
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
@@ -231008,76 +231214,82 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v0, v3, v0
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x30000, v0
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v61, vcc, 0x30000, v0
-; SI-NEXT: v_alignbit_b32 v0, v61, v3, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; SI-NEXT: v_add_i32_e32 v62, vcc, 0x30000, v0
+; SI-NEXT: v_alignbit_b32 v0, v62, v3, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v46, v5, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v63, v5, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:164 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v62, v7, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v60, v7, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:172 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v59, v9, 16
+; SI-NEXT: v_alignbit_b32 v0, v56, v9, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:180 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v57, v11, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v47, v8, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:184 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v56, v10, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v46, v10, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_alignbit_b32 v0, v45, v12, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:196 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_alignbit_b32 v0, v44, v14, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:176 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_alignbit_b32 v0, v43, v16, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:168 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_alignbit_b32 v0, v42, v18, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v41, v20, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v22, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v55, v22, 16
+; SI-NEXT: v_alignbit_b32 v0, v41, v2, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v53, v24, 16
+; SI-NEXT: v_alignbit_b32 v0, v40, v4, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:148 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v4, v26, 16
+; SI-NEXT: v_alignbit_b32 v0, v55, v24, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:144 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v2, v28, 16
+; SI-NEXT: v_alignbit_b32 v0, v26, v29, 16
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:140 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_alignbit_b32 v0, v6, v8, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v28, v32, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_alignbit_b32 v0, v6, v30, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v56
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v47
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v46
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v45
@@ -231095,38 +231307,43 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v41
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v55
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v40
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v53
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v55
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v4
-; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v61
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v26
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(1)
-; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v2
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v28
+; SI-NEXT: v_lshrrev_b32_e32 v58, 16, v62
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v6
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: .LBB110_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:164 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:172 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v58
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v63
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v60
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v40
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v54
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v61
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v57
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
@@ -231135,45 +231352,45 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v52
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v61
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v62
; SI-NEXT: v_or_b32_e32 v1, v1, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: s_waitcnt vmcnt(12)
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
; SI-NEXT: s_waitcnt vmcnt(9)
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: s_waitcnt vmcnt(8)
@@ -231190,44 +231407,45 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_or_b32_e32 v2, v2, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v46
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v63
; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v5
-; SI-NEXT: v_and_b32_e32 v5, 0xffff, v62
+; SI-NEXT: v_and_b32_e32 v5, 0xffff, v60
; SI-NEXT: v_or_b32_e32 v5, v5, v6
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_or_b32_e32 v6, v6, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v59
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v56
; SI-NEXT: v_or_b32_e32 v7, v7, v8
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_or_b32_e32 v8, v8, v9
-; SI-NEXT: v_and_b32_e32 v9, 0xffff, v57
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v47
; SI-NEXT: v_or_b32_e32 v9, v9, v10
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
; SI-NEXT: v_or_b32_e32 v10, v10, v11
-; SI-NEXT: v_and_b32_e32 v11, 0xffff, v56
+; SI-NEXT: v_and_b32_e32 v11, 0xffff, v46
; SI-NEXT: v_or_b32_e32 v11, v11, v12
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_or_b32_e32 v12, v12, v13
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v45
; SI-NEXT: v_or_b32_e32 v13, v13, v14
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_or_b32_e32 v14, v14, v15
@@ -231239,19 +231457,25 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v16, v16, v17
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v43
; SI-NEXT: v_or_b32_e32 v17, v17, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:160 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:168 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
; SI-NEXT: v_or_b32_e32 v18, v18, v19
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v42
; SI-NEXT: v_or_b32_e32 v19, v19, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:156 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v20, 0xffff, v20
; SI-NEXT: v_or_b32_e32 v20, v20, v21
; SI-NEXT: v_and_b32_e32 v21, 0xffff, v41
; SI-NEXT: v_or_b32_e32 v21, v21, v22
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
+; SI-NEXT: v_or_b32_e32 v22, v22, v23
+; SI-NEXT: v_and_b32_e32 v23, 0xffff, v40
+; SI-NEXT: v_or_b32_e32 v23, v23, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
@@ -231269,37 +231493,31 @@ define <64 x half> @bitcast_v64i16_to_v64f16(<64 x i16> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
-; SI-NEXT: v_or_b32_e32 v22, v22, v23
-; SI-NEXT: v_and_b32_e32 v23, 0xffff, v55
-; SI-NEXT: v_or_b32_e32 v23, v23, v24
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
; SI-NEXT: v_or_b32_e32 v24, v24, v25
-; SI-NEXT: v_and_b32_e32 v25, 0xffff, v53
+; SI-NEXT: v_and_b32_e32 v25, 0xffff, v55
; SI-NEXT: v_or_b32_e32 v25, v25, v26
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v26, 0xffff, v26
; SI-NEXT: v_or_b32_e32 v26, v26, v27
-; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v27, 0xffff, v27
; SI-NEXT: v_or_b32_e32 v27, v27, v28
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v28, 0xffff, v28
; SI-NEXT: v_or_b32_e32 v28, v28, v29
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v29, 0xffff, v29
; SI-NEXT: v_or_b32_e32 v29, v29, v30
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v30, 0xffff, v30
; SI-NEXT: v_or_b32_e32 v30, v30, v31
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v31, 0xffff, v31
; SI-NEXT: v_or_b32_e32 v31, v31, v32
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.256bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.256bit.ll
index d9e5e138fb87c..755c356474ebb 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.256bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.256bit.ll
@@ -41242,150 +41242,150 @@ define <32 x i8> @bitcast_v16bf16_to_v32i8(<16 x bfloat> %a, i32 %b) #0 {
; GFX9-NEXT: v_or_b32_e32 v4, 0x400000, v2
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v2, v2
; GFX9-NEXT: v_bfe_u32 v2, v1, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v4, v3, v4, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v9, v3, v4, vcc
; GFX9-NEXT: v_add3_u32 v2, v2, v1, s6
; GFX9-NEXT: v_or_b32_e32 v3, 0x400000, v1
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v0
-; GFX9-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
+; GFX9-NEXT: v_cndmask_b32_e32 v1, v2, v3, vcc
+; GFX9-NEXT: v_lshlrev_b32_e32 v2, 16, v0
+; GFX9-NEXT: v_add_f32_e32 v2, 0x40c00000, v2
; GFX9-NEXT: v_and_b32_e32 v0, 0xffff0000, v0
-; GFX9-NEXT: v_cndmask_b32_e32 v3, v2, v3, vcc
-; GFX9-NEXT: v_bfe_u32 v2, v1, 16, 1
+; GFX9-NEXT: v_bfe_u32 v4, v2, 16, 1
; GFX9-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
-; GFX9-NEXT: v_add3_u32 v2, v2, v1, s6
-; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v1
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: v_bfe_u32 v1, v0, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v2, v5, vcc
-; GFX9-NEXT: v_add3_u32 v1, v1, v0, s6
-; GFX9-NEXT: v_or_b32_e32 v2, 0x400000, v0
+; GFX9-NEXT: v_add3_u32 v4, v4, v2, s6
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v2
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v2, v2
+; GFX9-NEXT: v_bfe_u32 v2, v0, 16, 1
+; GFX9-NEXT: v_cndmask_b32_e32 v11, v4, v5, vcc
+; GFX9-NEXT: v_add3_u32 v2, v2, v0, s6
+; GFX9-NEXT: v_or_b32_e32 v4, 0x400000, v0
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 16, v36
; GFX9-NEXT: v_add_f32_e32 v0, 0x40c00000, v0
-; GFX9-NEXT: v_cndmask_b32_e32 v2, v1, v2, vcc
-; GFX9-NEXT: v_bfe_u32 v1, v0, 16, 1
-; GFX9-NEXT: v_add3_u32 v1, v1, v0, s6
-; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v0
+; GFX9-NEXT: v_cndmask_b32_e32 v8, v2, v4, vcc
+; GFX9-NEXT: v_bfe_u32 v4, v0, 16, 1
+; GFX9-NEXT: v_add3_u32 v4, v4, v0, s6
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v0
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
-; GFX9-NEXT: v_cndmask_b32_e32 v0, v1, v8, vcc
-; GFX9-NEXT: v_and_b32_e32 v1, 0xffff0000, v36
-; GFX9-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; GFX9-NEXT: v_bfe_u32 v8, v1, 16, 1
-; GFX9-NEXT: v_add3_u32 v8, v8, v1, s6
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v1
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v35
-; GFX9-NEXT: v_add_f32_e32 v1, 0x40c00000, v1
-; GFX9-NEXT: v_cndmask_b32_e32 v10, v8, v9, vcc
-; GFX9-NEXT: v_bfe_u32 v8, v1, 16, 1
-; GFX9-NEXT: v_add3_u32 v8, v8, v1, s6
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v1
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: v_cndmask_b32_e32 v1, v8, v9, vcc
-; GFX9-NEXT: v_and_b32_e32 v8, 0xffff0000, v35
-; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_bfe_u32 v9, v8, 16, 1
-; GFX9-NEXT: v_add3_u32 v9, v9, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v11, 0x400000, v8
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v33
-; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v13, v9, v11, vcc
-; GFX9-NEXT: v_bfe_u32 v9, v8, 16, 1
-; GFX9-NEXT: v_add3_u32 v9, v9, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v11, 0x400000, v8
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_and_b32_e32 v8, 0xffff0000, v33
-; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v14, v9, v11, vcc
-; GFX9-NEXT: v_bfe_u32 v9, v8, 16, 1
-; GFX9-NEXT: v_add3_u32 v9, v9, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v11, 0x400000, v8
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v32
-; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v15, v9, v11, vcc
-; GFX9-NEXT: v_bfe_u32 v11, v8, 16, 1
-; GFX9-NEXT: v_add3_u32 v11, v11, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v12, 0x400000, v8
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_and_b32_e32 v8, 0xffff0000, v32
-; GFX9-NEXT: v_add_f32_e32 v8, 0x40c00000, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v16, v11, v12, vcc
-; GFX9-NEXT: v_bfe_u32 v11, v8, 16, 1
-; GFX9-NEXT: v_add3_u32 v11, v11, v8, s6
-; GFX9-NEXT: v_or_b32_e32 v12, 0x400000, v8
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
-; GFX9-NEXT: v_cndmask_b32_e32 v17, v11, v12, vcc
-; GFX9-NEXT: v_lshlrev_b32_e32 v11, 16, v7
-; GFX9-NEXT: v_add_f32_e32 v11, 0x40c00000, v11
+; GFX9-NEXT: v_cndmask_b32_e32 v0, v4, v5, vcc
+; GFX9-NEXT: v_and_b32_e32 v4, 0xffff0000, v36
+; GFX9-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; GFX9-NEXT: v_bfe_u32 v5, v4, 16, 1
+; GFX9-NEXT: v_add3_u32 v5, v5, v4, s6
+; GFX9-NEXT: v_or_b32_e32 v10, 0x400000, v4
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
+; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v35
+; GFX9-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; GFX9-NEXT: v_bfe_u32 v12, v4, 16, 1
+; GFX9-NEXT: v_cndmask_b32_e32 v10, v5, v10, vcc
+; GFX9-NEXT: v_add3_u32 v12, v12, v4, s6
+; GFX9-NEXT: v_or_b32_e32 v13, 0x400000, v4
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
+; GFX9-NEXT: v_and_b32_e32 v4, 0xffff0000, v35
+; GFX9-NEXT: v_add_f32_e32 v4, 0x40c00000, v4
+; GFX9-NEXT: v_cndmask_b32_e32 v17, v12, v13, vcc
+; GFX9-NEXT: v_bfe_u32 v12, v4, 16, 1
+; GFX9-NEXT: v_add3_u32 v12, v12, v4, s6
+; GFX9-NEXT: v_or_b32_e32 v13, 0x400000, v4
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
+; GFX9-NEXT: v_cndmask_b32_e32 v19, v12, v13, vcc
+; GFX9-NEXT: v_lshlrev_b32_e32 v12, 16, v33
+; GFX9-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
+; GFX9-NEXT: v_bfe_u32 v13, v12, 16, 1
+; GFX9-NEXT: v_add3_u32 v13, v13, v12, s6
+; GFX9-NEXT: v_or_b32_e32 v14, 0x400000, v12
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
+; GFX9-NEXT: v_and_b32_e32 v12, 0xffff0000, v33
+; GFX9-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
+; GFX9-NEXT: v_cndmask_b32_e32 v14, v13, v14, vcc
+; GFX9-NEXT: v_bfe_u32 v13, v12, 16, 1
+; GFX9-NEXT: v_add3_u32 v13, v13, v12, s6
+; GFX9-NEXT: v_or_b32_e32 v15, 0x400000, v12
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
+; GFX9-NEXT: v_lshlrev_b32_e32 v12, 16, v32
+; GFX9-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
+; GFX9-NEXT: v_cndmask_b32_e32 v18, v13, v15, vcc
+; GFX9-NEXT: v_bfe_u32 v15, v12, 16, 1
+; GFX9-NEXT: v_add3_u32 v15, v15, v12, s6
+; GFX9-NEXT: v_or_b32_e32 v16, 0x400000, v12
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
+; GFX9-NEXT: v_and_b32_e32 v12, 0xffff0000, v32
+; GFX9-NEXT: v_add_f32_e32 v12, 0x40c00000, v12
+; GFX9-NEXT: v_cndmask_b32_e32 v20, v15, v16, vcc
+; GFX9-NEXT: v_bfe_u32 v15, v12, 16, 1
+; GFX9-NEXT: v_add3_u32 v15, v15, v12, s6
+; GFX9-NEXT: v_or_b32_e32 v16, 0x400000, v12
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
+; GFX9-NEXT: v_cndmask_b32_e32 v21, v15, v16, vcc
+; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v7
+; GFX9-NEXT: v_add_f32_e32 v15, 0x40c00000, v15
; GFX9-NEXT: v_and_b32_e32 v7, 0xffff0000, v7
-; GFX9-NEXT: v_bfe_u32 v12, v11, 16, 1
+; GFX9-NEXT: v_bfe_u32 v16, v15, 16, 1
; GFX9-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
-; GFX9-NEXT: v_add3_u32 v12, v12, v11, s6
-; GFX9-NEXT: v_or_b32_e32 v18, 0x400000, v11
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v11, v11
-; GFX9-NEXT: v_bfe_u32 v11, v7, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v18, v12, v18, vcc
-; GFX9-NEXT: v_add3_u32 v11, v11, v7, s6
-; GFX9-NEXT: v_or_b32_e32 v12, 0x400000, v7
+; GFX9-NEXT: v_add3_u32 v16, v16, v15, s6
+; GFX9-NEXT: v_or_b32_e32 v22, 0x400000, v15
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v15, v15
+; GFX9-NEXT: v_bfe_u32 v15, v7, 16, 1
+; GFX9-NEXT: v_cndmask_b32_e32 v22, v16, v22, vcc
+; GFX9-NEXT: v_add3_u32 v15, v15, v7, s6
+; GFX9-NEXT: v_or_b32_e32 v16, 0x400000, v7
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
-; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v6
-; GFX9-NEXT: v_add_f32_e32 v7, 0x40c00000, v7
+; GFX9-NEXT: v_cndmask_b32_e32 v7, v15, v16, vcc
+; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v6
+; GFX9-NEXT: v_add_f32_e32 v15, 0x40c00000, v15
; GFX9-NEXT: v_and_b32_e32 v6, 0xffff0000, v6
-; GFX9-NEXT: v_cndmask_b32_e32 v19, v11, v12, vcc
-; GFX9-NEXT: v_bfe_u32 v11, v7, 16, 1
+; GFX9-NEXT: v_bfe_u32 v23, v15, 16, 1
; GFX9-NEXT: v_add_f32_e32 v6, 0x40c00000, v6
-; GFX9-NEXT: v_add3_u32 v11, v11, v7, s6
-; GFX9-NEXT: v_or_b32_e32 v12, 0x400000, v7
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
-; GFX9-NEXT: v_bfe_u32 v7, v6, 16, 1
-; GFX9-NEXT: v_cndmask_b32_e32 v20, v11, v12, vcc
-; GFX9-NEXT: v_add3_u32 v7, v7, v6, s6
-; GFX9-NEXT: v_or_b32_e32 v11, 0x400000, v6
+; GFX9-NEXT: v_add3_u32 v23, v23, v15, s6
+; GFX9-NEXT: v_or_b32_e32 v24, 0x400000, v15
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v15, v15
+; GFX9-NEXT: v_bfe_u32 v15, v6, 16, 1
+; GFX9-NEXT: v_cndmask_b32_e32 v23, v23, v24, vcc
+; GFX9-NEXT: v_add3_u32 v15, v15, v6, s6
+; GFX9-NEXT: v_or_b32_e32 v24, 0x400000, v6
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v6, v6
; GFX9-NEXT: s_mov_b32 s7, 0x7060302
-; GFX9-NEXT: v_cndmask_b32_e32 v21, v7, v11, vcc
-; GFX9-NEXT: v_perm_b32 v9, v15, v14, s7
-; GFX9-NEXT: v_perm_b32 v8, v17, v16, s7
-; GFX9-NEXT: v_perm_b32 v12, v19, v18, s7
-; GFX9-NEXT: v_perm_b32 v11, v21, v20, s7
-; GFX9-NEXT: v_perm_b32 v38, v10, v0, s7
-; GFX9-NEXT: v_perm_b32 v37, v13, v1, s7
-; GFX9-NEXT: v_perm_b32 v49, v3, v4, s7
-; GFX9-NEXT: v_perm_b32 v48, v2, v5, s7
-; GFX9-NEXT: v_lshrrev_b32_e32 v6, 16, v20
-; GFX9-NEXT: v_lshrrev_b32_e32 v35, 16, v1
-; GFX9-NEXT: v_lshrrev_b32_e32 v1, 16, v4
-; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v19
-; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v3
-; GFX9-NEXT: v_lshrrev_b32_e32 v31, 24, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v29, 8, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v25, 8, v11
-; GFX9-NEXT: v_lshrrev_b64 v[27:28], 24, v[11:12]
-; GFX9-NEXT: v_lshrrev_b64 v[19:20], 24, v[8:9]
-; GFX9-NEXT: v_lshrrev_b64 v[11:12], 24, v[37:38]
-; GFX9-NEXT: v_lshrrev_b64 v[3:4], 24, v[48:49]
-; GFX9-NEXT: v_lshrrev_b32_e32 v7, 16, v18
+; GFX9-NEXT: v_cndmask_b32_e32 v6, v15, v24, vcc
+; GFX9-NEXT: v_perm_b32 v3, v1, v9, s7
+; GFX9-NEXT: v_perm_b32 v2, v8, v11, s7
+; GFX9-NEXT: v_perm_b32 v5, v10, v0, s7
+; GFX9-NEXT: v_perm_b32 v4, v19, v17, s7
+; GFX9-NEXT: v_perm_b32 v13, v18, v14, s7
+; GFX9-NEXT: v_perm_b32 v12, v21, v20, s7
+; GFX9-NEXT: v_perm_b32 v16, v7, v22, s7
+; GFX9-NEXT: v_perm_b32 v15, v6, v23, s7
+; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v7
+; GFX9-NEXT: v_lshrrev_b32_e32 v7, 16, v22
+; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v6, 16, v23
+; GFX9-NEXT: v_lshrrev_b32_e32 v22, 16, v18
; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v14
-; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v16
-; GFX9-NEXT: v_lshrrev_b32_e32 v36, 16, v0
-; GFX9-NEXT: v_lshrrev_b32_e32 v0, 16, v5
-; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v21
-; GFX9-NEXT: v_lshrrev_b32_e32 v22, 16, v15
-; GFX9-NEXT: v_lshrrev_b32_e32 v18, 16, v17
+; GFX9-NEXT: v_lshrrev_b32_e32 v18, 16, v21
+; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v20
; GFX9-NEXT: v_lshrrev_b32_e32 v14, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v10, 16, v13
-; GFX9-NEXT: v_lshrrev_b32_e32 v23, 24, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v21, 8, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v17, 8, v8
-; GFX9-NEXT: v_lshrrev_b32_e32 v15, 24, v38
-; GFX9-NEXT: v_lshrrev_b32_e32 v13, 8, v38
-; GFX9-NEXT: v_lshrrev_b32_e32 v9, 8, v37
-; GFX9-NEXT: v_lshrrev_b32_e32 v37, 24, v49
-; GFX9-NEXT: v_lshrrev_b32_e32 v5, 8, v49
-; GFX9-NEXT: v_lshrrev_b32_e32 v38, 8, v48
-; GFX9-NEXT: v_lshrrev_b32_e32 v2, 16, v2
+; GFX9-NEXT: v_lshrrev_b32_e32 v36, 16, v0
+; GFX9-NEXT: v_lshrrev_b32_e32 v10, 16, v19
+; GFX9-NEXT: v_lshrrev_b32_e32 v35, 16, v17
+; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v1
+; GFX9-NEXT: v_lshrrev_b32_e32 v1, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v0, 16, v11
+; GFX9-NEXT: v_lshrrev_b64 v[27:28], 24, v[15:16]
+; GFX9-NEXT: v_lshrrev_b64 v[19:20], 24, v[12:13]
+; GFX9-NEXT: v_lshrrev_b32_e32 v25, 8, v15
+; GFX9-NEXT: v_lshrrev_b32_e32 v23, 24, v13
+; GFX9-NEXT: v_lshrrev_b32_e32 v21, 8, v13
+; GFX9-NEXT: v_lshrrev_b32_e32 v17, 8, v12
+; GFX9-NEXT: v_lshrrev_b32_e32 v15, 24, v5
+; GFX9-NEXT: v_lshrrev_b32_e32 v13, 8, v5
+; GFX9-NEXT: v_lshrrev_b32_e32 v9, 8, v4
+; GFX9-NEXT: v_lshrrev_b64 v[11:12], 24, v[4:5]
+; GFX9-NEXT: v_lshrrev_b32_e32 v37, 24, v3
+; GFX9-NEXT: v_lshrrev_b32_e32 v5, 8, v3
+; GFX9-NEXT: v_lshrrev_b64 v[3:4], 24, v[2:3]
+; GFX9-NEXT: v_lshrrev_b32_e32 v31, 24, v16
+; GFX9-NEXT: v_lshrrev_b32_e32 v29, 8, v16
+; GFX9-NEXT: v_lshrrev_b32_e32 v38, 8, v2
+; GFX9-NEXT: v_lshrrev_b32_e32 v2, 16, v8
; GFX9-NEXT: .LBB108_4: ; %end
; GFX9-NEXT: s_or_b64 exec, exec, s[4:5]
; GFX9-NEXT: v_mov_b32_e32 v4, v1
@@ -42483,7 +42483,7 @@ define inreg <32 x i8> @bitcast_v16bf16_to_v32i8_scalar(<16 x bfloat> inreg %a,
; GFX9-NEXT: v_lshrrev_b32_e32 v35, 16, v0
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 16, v6
; GFX9-NEXT: s_and_b32 s4, s16, 0xffff0000
-; GFX9-NEXT: v_or_b32_sdwa v4, v0, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v37, v0, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_add_f32_e32 v0, s4, v1
; GFX9-NEXT: v_bfe_u32 v2, v0, 16, 1
; GFX9-NEXT: v_add_u32_e32 v2, v2, v0
@@ -42497,148 +42497,148 @@ define inreg <32 x i8> @bitcast_v16bf16_to_v32i8_scalar(<16 x bfloat> inreg %a,
; GFX9-NEXT: v_bfe_u32 v3, v0, 16, 1
; GFX9-NEXT: v_add_u32_e32 v3, v3, v0
; GFX9-NEXT: v_add_u32_e32 v3, 0x7fff, v3
-; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v0
+; GFX9-NEXT: v_or_b32_e32 v4, 0x400000, v0
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
+; GFX9-NEXT: v_cndmask_b32_e32 v0, v3, v4, vcc
+; GFX9-NEXT: v_lshrrev_b32_e32 v0, 16, v0
+; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v2
; GFX9-NEXT: s_and_b32 s4, s19, 0xffff0000
-; GFX9-NEXT: v_cndmask_b32_e32 v0, v3, v5, vcc
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v8, vcc
+; GFX9-NEXT: v_or_b32_sdwa v36, v3, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
; GFX9-NEXT: s_lshl_b32 s4, s19, 16
-; GFX9-NEXT: v_lshrrev_b32_e32 v14, 16, v5
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v8, vcc
-; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v5
-; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v14
+; GFX9-NEXT: v_lshrrev_b32_e32 v14, 16, v3
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
+; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v3
+; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v14
; GFX9-NEXT: s_and_b32 s4, s18, 0xffff0000
-; GFX9-NEXT: v_or_b32_sdwa v12, v5, v34 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v8, vcc
+; GFX9-NEXT: v_or_b32_sdwa v39, v3, v34 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
; GFX9-NEXT: s_lshl_b32 s4, s18, 16
-; GFX9-NEXT: v_lshrrev_b32_e32 v10, 16, v5
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v8, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v8, vcc
-; GFX9-NEXT: v_lshrrev_b32_e32 v8, 16, v5
-; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v10
+; GFX9-NEXT: v_lshrrev_b32_e32 v10, 16, v3
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
+; GFX9-NEXT: v_lshrrev_b32_e32 v8, 16, v3
+; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v10
; GFX9-NEXT: s_and_b32 s4, s21, 0xffff0000
-; GFX9-NEXT: v_or_b32_sdwa v11, v5, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v9, vcc
+; GFX9-NEXT: v_or_b32_sdwa v38, v3, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
; GFX9-NEXT: s_lshl_b32 s4, s21, 16
-; GFX9-NEXT: v_lshrrev_b32_e32 v22, 16, v5
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v9, vcc
-; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v5
-; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v22
+; GFX9-NEXT: v_lshrrev_b32_e32 v22, 16, v3
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
+; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v3
+; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v22
; GFX9-NEXT: s_and_b32 s4, s20, 0xffff0000
-; GFX9-NEXT: v_or_b32_sdwa v20, v5, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v9, vcc
+; GFX9-NEXT: v_or_b32_sdwa v49, v3, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
; GFX9-NEXT: s_lshl_b32 s4, s20, 16
-; GFX9-NEXT: v_lshrrev_b32_e32 v18, 16, v5
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v9, vcc
-; GFX9-NEXT: v_lshrrev_b32_e32 v16, 16, v5
-; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v18
+; GFX9-NEXT: v_lshrrev_b32_e32 v18, 16, v3
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
+; GFX9-NEXT: v_lshrrev_b32_e32 v16, 16, v3
+; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v18
; GFX9-NEXT: s_and_b32 s4, s23, 0xffff0000
-; GFX9-NEXT: v_or_b32_sdwa v19, v5, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_add_f32_e32 v5, s4, v1
-; GFX9-NEXT: v_bfe_u32 v7, v5, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v7, v7, v5
-; GFX9-NEXT: v_add_u32_e32 v7, 0x7fff, v7
-; GFX9-NEXT: v_or_b32_e32 v9, 0x400000, v5
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
+; GFX9-NEXT: v_or_b32_sdwa v48, v3, v16 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
; GFX9-NEXT: s_lshl_b32 s4, s23, 16
-; GFX9-NEXT: v_cndmask_b32_e32 v5, v7, v9, vcc
-; GFX9-NEXT: v_add_f32_e32 v7, s4, v1
-; GFX9-NEXT: v_bfe_u32 v9, v7, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v9, v9, v7
-; GFX9-NEXT: v_add_u32_e32 v9, 0x7fff, v9
-; GFX9-NEXT: v_or_b32_e32 v13, 0x400000, v7
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
+; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v3
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
+; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v3
+; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v30
; GFX9-NEXT: s_and_b32 s4, s22, 0xffff0000
-; GFX9-NEXT: v_cndmask_b32_e32 v7, v9, v13, vcc
-; GFX9-NEXT: v_add_f32_e32 v9, s4, v1
-; GFX9-NEXT: v_bfe_u32 v13, v9, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v13, v13, v9
+; GFX9-NEXT: v_or_b32_sdwa v51, v3, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_add_f32_e32 v3, s4, v1
+; GFX9-NEXT: v_bfe_u32 v4, v3, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v4, v4, v3
+; GFX9-NEXT: v_add_u32_e32 v4, 0x7fff, v4
+; GFX9-NEXT: v_or_b32_e32 v5, 0x400000, v3
+; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
; GFX9-NEXT: s_lshl_b32 s4, s22, 16
-; GFX9-NEXT: v_add_u32_e32 v13, 0x7fff, v13
-; GFX9-NEXT: v_or_b32_e32 v15, 0x400000, v9
-; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v9, v9
+; GFX9-NEXT: v_cndmask_b32_e32 v3, v4, v5, vcc
; GFX9-NEXT: v_add_f32_e32 v1, s4, v1
-; GFX9-NEXT: v_cndmask_b32_e32 v9, v13, v15, vcc
-; GFX9-NEXT: v_bfe_u32 v13, v1, 16, 1
-; GFX9-NEXT: v_add_u32_e32 v13, v13, v1
-; GFX9-NEXT: v_add_u32_e32 v13, 0x7fff, v13
-; GFX9-NEXT: v_or_b32_e32 v15, 0x400000, v1
+; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v3
+; GFX9-NEXT: v_bfe_u32 v3, v1, 16, 1
+; GFX9-NEXT: v_add_u32_e32 v3, v3, v1
+; GFX9-NEXT: v_add_u32_e32 v3, 0x7fff, v3
+; GFX9-NEXT: v_or_b32_e32 v4, 0x400000, v1
; GFX9-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
-; GFX9-NEXT: v_cndmask_b32_e32 v1, v13, v15, vcc
-; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v5
-; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v9
+; GFX9-NEXT: v_cndmask_b32_e32 v1, v3, v4, vcc
; GFX9-NEXT: v_lshrrev_b32_e32 v24, 16, v1
-; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v30
-; GFX9-NEXT: v_lshrrev_b32_e32 v0, 16, v0
-; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v2
-; GFX9-NEXT: v_or_b32_sdwa v28, v1, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v26
-; GFX9-NEXT: v_or_b32_sdwa v3, v3, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v27, v1, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshrrev_b32_e32 v31, 24, v28
-; GFX9-NEXT: v_lshrrev_b32_e32 v29, 8, v28
-; GFX9-NEXT: v_lshrrev_b32_e32 v25, 8, v27
-; GFX9-NEXT: v_lshrrev_b64 v[27:28], 24, v[27:28]
-; GFX9-NEXT: v_lshrrev_b32_e32 v23, 24, v20
-; GFX9-NEXT: v_lshrrev_b32_e32 v21, 8, v20
-; GFX9-NEXT: v_lshrrev_b32_e32 v17, 8, v19
-; GFX9-NEXT: v_lshrrev_b64 v[19:20], 24, v[19:20]
-; GFX9-NEXT: v_lshrrev_b32_e32 v15, 24, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v13, 8, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v9, 8, v11
-; GFX9-NEXT: v_lshrrev_b64 v[11:12], 24, v[11:12]
-; GFX9-NEXT: v_lshrrev_b32_e32 v7, 24, v4
-; GFX9-NEXT: v_lshrrev_b32_e32 v5, 8, v4
-; GFX9-NEXT: v_lshrrev_b32_e32 v1, 8, v3
-; GFX9-NEXT: v_lshrrev_b64 v[3:4], 24, v[3:4]
+; GFX9-NEXT: v_or_b32_sdwa v50, v1, v24 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshrrev_b64 v[27:28], 24, v[50:51]
+; GFX9-NEXT: v_lshrrev_b64 v[19:20], 24, v[48:49]
+; GFX9-NEXT: v_lshrrev_b64 v[11:12], 24, v[38:39]
+; GFX9-NEXT: v_lshrrev_b64 v[3:4], 24, v[36:37]
+; GFX9-NEXT: v_lshrrev_b32_e32 v31, 24, v51
+; GFX9-NEXT: v_lshrrev_b32_e32 v29, 8, v51
+; GFX9-NEXT: v_lshrrev_b32_e32 v25, 8, v50
+; GFX9-NEXT: v_lshrrev_b32_e32 v23, 24, v49
+; GFX9-NEXT: v_lshrrev_b32_e32 v21, 8, v49
+; GFX9-NEXT: v_lshrrev_b32_e32 v17, 8, v48
+; GFX9-NEXT: v_lshrrev_b32_e32 v15, 24, v39
+; GFX9-NEXT: v_lshrrev_b32_e32 v13, 8, v39
+; GFX9-NEXT: v_lshrrev_b32_e32 v9, 8, v38
+; GFX9-NEXT: v_lshrrev_b32_e32 v7, 24, v37
+; GFX9-NEXT: v_lshrrev_b32_e32 v5, 8, v37
+; GFX9-NEXT: v_lshrrev_b32_e32 v1, 8, v36
; GFX9-NEXT: s_branch .LBB109_6
; GFX9-NEXT: .LBB109_5:
; GFX9-NEXT: v_mov_b32_e32 v24, s22
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.384bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.384bit.ll
index 0527d0739ffbe..c5406de10bce5 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.384bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.384bit.ll
@@ -5406,12 +5406,12 @@ define inreg <24 x i16> @bitcast_v12f32_to_v24i16_scalar(<12 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[16:17], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[17:18], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v1
; SI-NEXT: s_branch .LBB29_6
; SI-NEXT: .LBB29_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -5426,12 +5426,12 @@ define inreg <24 x i16> @bitcast_v12f32_to_v24i16_scalar(<12 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v9, s25
; SI-NEXT: v_mov_b32_e32 v10, s26
; SI-NEXT: v_mov_b32_e32 v11, s27
-; SI-NEXT: v_mov_b32_e32 v24, s40
-; SI-NEXT: v_mov_b32_e32 v23, s41
-; SI-NEXT: v_mov_b32_e32 v22, s42
-; SI-NEXT: v_mov_b32_e32 v21, s43
-; SI-NEXT: v_mov_b32_e32 v20, s44
-; SI-NEXT: v_mov_b32_e32 v19, s45
+; SI-NEXT: v_mov_b32_e32 v23, s40
+; SI-NEXT: v_mov_b32_e32 v22, s41
+; SI-NEXT: v_mov_b32_e32 v21, s42
+; SI-NEXT: v_mov_b32_e32 v20, s43
+; SI-NEXT: v_mov_b32_e32 v19, s44
+; SI-NEXT: v_mov_b32_e32 v18, s45
; SI-NEXT: v_mov_b32_e32 v17, s14
; SI-NEXT: v_mov_b32_e32 v16, s12
; SI-NEXT: v_mov_b32_e32 v15, s10
@@ -5447,28 +5447,28 @@ define inreg <24 x i16> @bitcast_v12f32_to_v24i16_scalar(<12 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; SI-NEXT: v_or_b32_e32 v0, v0, v17
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v23
; SI-NEXT: v_or_b32_e32 v2, v2, v16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v23
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v22
; SI-NEXT: v_or_b32_e32 v4, v4, v15
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v21
; SI-NEXT: v_or_b32_e32 v6, v6, v14
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v21
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v20
; SI-NEXT: v_or_b32_e32 v8, v8, v13
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v20
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v19
; SI-NEXT: v_or_b32_e32 v10, v10, v12
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v19
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v18
; SI-NEXT: v_or_b32_e32 v1, v1, v17
; SI-NEXT: v_or_b32_e32 v3, v3, v16
; SI-NEXT: v_or_b32_e32 v5, v5, v15
@@ -6481,12 +6481,12 @@ define inreg <24 x half> @bitcast_v12f32_to_v24f16_scalar(<12 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[16:17], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[17:18], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v1
; SI-NEXT: s_branch .LBB33_6
; SI-NEXT: .LBB33_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -6501,12 +6501,12 @@ define inreg <24 x half> @bitcast_v12f32_to_v24f16_scalar(<12 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v9, s25
; SI-NEXT: v_mov_b32_e32 v10, s26
; SI-NEXT: v_mov_b32_e32 v11, s27
-; SI-NEXT: v_mov_b32_e32 v24, s40
-; SI-NEXT: v_mov_b32_e32 v23, s41
-; SI-NEXT: v_mov_b32_e32 v22, s42
-; SI-NEXT: v_mov_b32_e32 v21, s43
-; SI-NEXT: v_mov_b32_e32 v20, s44
-; SI-NEXT: v_mov_b32_e32 v19, s45
+; SI-NEXT: v_mov_b32_e32 v23, s40
+; SI-NEXT: v_mov_b32_e32 v22, s41
+; SI-NEXT: v_mov_b32_e32 v21, s42
+; SI-NEXT: v_mov_b32_e32 v20, s43
+; SI-NEXT: v_mov_b32_e32 v19, s44
+; SI-NEXT: v_mov_b32_e32 v18, s45
; SI-NEXT: v_mov_b32_e32 v17, s14
; SI-NEXT: v_mov_b32_e32 v16, s12
; SI-NEXT: v_mov_b32_e32 v15, s10
@@ -6522,28 +6522,28 @@ define inreg <24 x half> @bitcast_v12f32_to_v24f16_scalar(<12 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; SI-NEXT: v_or_b32_e32 v0, v0, v17
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v23
; SI-NEXT: v_or_b32_e32 v2, v2, v16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v23
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v22
; SI-NEXT: v_or_b32_e32 v4, v4, v15
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v21
; SI-NEXT: v_or_b32_e32 v6, v6, v14
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v21
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v20
; SI-NEXT: v_or_b32_e32 v8, v8, v13
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v20
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v19
; SI-NEXT: v_or_b32_e32 v10, v10, v12
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v19
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v18
; SI-NEXT: v_or_b32_e32 v1, v1, v17
; SI-NEXT: v_or_b32_e32 v3, v3, v16
; SI-NEXT: v_or_b32_e32 v5, v5, v15
@@ -7984,14 +7984,14 @@ define <24 x i16> @bitcast_v6f64_to_v24i16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr23
-; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr15
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr14
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr13
-; SI-NEXT: ; implicit-def: $vgpr17
+; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr12
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
@@ -8002,10 +8002,10 @@ define <24 x i16> @bitcast_v6f64_to_v24i16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v13, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v14, v7, v6, 16
; SI-NEXT: v_alignbit_b32 v15, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v18, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v17, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v20, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v3
@@ -8024,10 +8024,10 @@ define <24 x i16> @bitcast_v6f64_to_v24i16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v13, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v14, v7, v6, 16
; SI-NEXT: v_alignbit_b32 v15, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v18, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v17, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v20, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v3
@@ -8036,22 +8036,22 @@ define <24 x i16> @bitcast_v6f64_to_v24i16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; SI-NEXT: v_or_b32_e32 v0, v0, v20
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v23
-; SI-NEXT: v_or_b32_e32 v2, v2, v18
+; SI-NEXT: v_or_b32_e32 v2, v2, v17
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v22
; SI-NEXT: v_or_b32_e32 v4, v4, v15
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v21
@@ -8060,12 +8060,12 @@ define <24 x i16> @bitcast_v6f64_to_v24i16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v19
; SI-NEXT: v_or_b32_e32 v8, v8, v13
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v17
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v18
; SI-NEXT: v_or_b32_e32 v10, v10, v12
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v16
; SI-NEXT: v_or_b32_e32 v1, v1, v20
-; SI-NEXT: v_or_b32_e32 v3, v3, v18
+; SI-NEXT: v_or_b32_e32 v3, v3, v17
; SI-NEXT: v_or_b32_e32 v5, v5, v15
; SI-NEXT: v_or_b32_e32 v7, v7, v14
; SI-NEXT: v_or_b32_e32 v9, v9, v13
@@ -9048,14 +9048,14 @@ define <24 x half> @bitcast_v6f64_to_v24f16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr23
-; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr15
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr14
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr13
-; SI-NEXT: ; implicit-def: $vgpr17
+; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr12
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
@@ -9066,10 +9066,10 @@ define <24 x half> @bitcast_v6f64_to_v24f16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v13, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v14, v7, v6, 16
; SI-NEXT: v_alignbit_b32 v15, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v18, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v17, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v20, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v3
@@ -9088,10 +9088,10 @@ define <24 x half> @bitcast_v6f64_to_v24f16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v13, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v14, v7, v6, 16
; SI-NEXT: v_alignbit_b32 v15, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v18, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v17, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v20, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v3
@@ -9100,22 +9100,22 @@ define <24 x half> @bitcast_v6f64_to_v24f16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; SI-NEXT: v_or_b32_e32 v0, v0, v20
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v23
-; SI-NEXT: v_or_b32_e32 v2, v2, v18
+; SI-NEXT: v_or_b32_e32 v2, v2, v17
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v22
; SI-NEXT: v_or_b32_e32 v4, v4, v15
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v21
@@ -9124,12 +9124,12 @@ define <24 x half> @bitcast_v6f64_to_v24f16(<6 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v19
; SI-NEXT: v_or_b32_e32 v8, v8, v13
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v17
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v18
; SI-NEXT: v_or_b32_e32 v10, v10, v12
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v16
; SI-NEXT: v_or_b32_e32 v1, v1, v20
-; SI-NEXT: v_or_b32_e32 v3, v3, v18
+; SI-NEXT: v_or_b32_e32 v3, v3, v17
; SI-NEXT: v_or_b32_e32 v5, v5, v15
; SI-NEXT: v_or_b32_e32 v7, v7, v14
; SI-NEXT: v_or_b32_e32 v9, v9, v13
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.448bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.448bit.ll
index 5950507d70a97..c332e65639c16 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.448bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.448bit.ll
@@ -6008,13 +6008,13 @@ define inreg <28 x i16> @bitcast_v14f32_to_v28i16_scalar(<14 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[19:20], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[20:21], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v1
; SI-NEXT: s_branch .LBB29_6
; SI-NEXT: .LBB29_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -6031,13 +6031,13 @@ define inreg <28 x i16> @bitcast_v14f32_to_v28i16_scalar(<14 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v11, s27
; SI-NEXT: v_mov_b32_e32 v12, s28
; SI-NEXT: v_mov_b32_e32 v13, s29
-; SI-NEXT: v_mov_b32_e32 v28, s44
-; SI-NEXT: v_mov_b32_e32 v27, s45
-; SI-NEXT: v_mov_b32_e32 v26, s46
-; SI-NEXT: v_mov_b32_e32 v25, s47
-; SI-NEXT: v_mov_b32_e32 v24, s56
-; SI-NEXT: v_mov_b32_e32 v23, s57
-; SI-NEXT: v_mov_b32_e32 v22, s58
+; SI-NEXT: v_mov_b32_e32 v27, s44
+; SI-NEXT: v_mov_b32_e32 v26, s45
+; SI-NEXT: v_mov_b32_e32 v25, s46
+; SI-NEXT: v_mov_b32_e32 v24, s47
+; SI-NEXT: v_mov_b32_e32 v23, s56
+; SI-NEXT: v_mov_b32_e32 v22, s57
+; SI-NEXT: v_mov_b32_e32 v21, s58
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s6
; SI-NEXT: v_mov_b32_e32 v16, s8
@@ -6054,33 +6054,33 @@ define inreg <28 x i16> @bitcast_v14f32_to_v28i16_scalar(<14 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v0, v0, v20
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
; SI-NEXT: v_or_b32_e32 v2, v2, v19
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v27
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v26
; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
; SI-NEXT: v_or_b32_e32 v6, v6, v17
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v24
; SI-NEXT: v_or_b32_e32 v8, v8, v16
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v23
; SI-NEXT: v_or_b32_e32 v10, v10, v15
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v23
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v22
; SI-NEXT: v_or_b32_e32 v12, v12, v14
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v21
; SI-NEXT: v_or_b32_e32 v1, v1, v20
; SI-NEXT: v_or_b32_e32 v3, v3, v19
; SI-NEXT: v_or_b32_e32 v5, v5, v18
@@ -7231,13 +7231,13 @@ define inreg <28 x half> @bitcast_v14f32_to_v28f16_scalar(<14 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[19:20], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[20:21], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v1
; SI-NEXT: s_branch .LBB33_6
; SI-NEXT: .LBB33_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -7254,13 +7254,13 @@ define inreg <28 x half> @bitcast_v14f32_to_v28f16_scalar(<14 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v11, s27
; SI-NEXT: v_mov_b32_e32 v12, s28
; SI-NEXT: v_mov_b32_e32 v13, s29
-; SI-NEXT: v_mov_b32_e32 v28, s44
-; SI-NEXT: v_mov_b32_e32 v27, s45
-; SI-NEXT: v_mov_b32_e32 v26, s46
-; SI-NEXT: v_mov_b32_e32 v25, s47
-; SI-NEXT: v_mov_b32_e32 v24, s56
-; SI-NEXT: v_mov_b32_e32 v23, s57
-; SI-NEXT: v_mov_b32_e32 v22, s58
+; SI-NEXT: v_mov_b32_e32 v27, s44
+; SI-NEXT: v_mov_b32_e32 v26, s45
+; SI-NEXT: v_mov_b32_e32 v25, s46
+; SI-NEXT: v_mov_b32_e32 v24, s47
+; SI-NEXT: v_mov_b32_e32 v23, s56
+; SI-NEXT: v_mov_b32_e32 v22, s57
+; SI-NEXT: v_mov_b32_e32 v21, s58
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s6
; SI-NEXT: v_mov_b32_e32 v16, s8
@@ -7277,33 +7277,33 @@ define inreg <28 x half> @bitcast_v14f32_to_v28f16_scalar(<14 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v0, v0, v20
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
; SI-NEXT: v_or_b32_e32 v2, v2, v19
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v27
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v26
; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
; SI-NEXT: v_or_b32_e32 v6, v6, v17
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v24
; SI-NEXT: v_or_b32_e32 v8, v8, v16
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v23
; SI-NEXT: v_or_b32_e32 v10, v10, v15
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v23
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v22
; SI-NEXT: v_or_b32_e32 v12, v12, v14
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v21
; SI-NEXT: v_or_b32_e32 v1, v1, v20
; SI-NEXT: v_or_b32_e32 v3, v3, v19
; SI-NEXT: v_or_b32_e32 v5, v5, v18
@@ -11435,7 +11435,7 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr26
-; SI-NEXT: ; implicit-def: $vgpr19
+; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr23
@@ -11444,7 +11444,7 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr15
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr14
-; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB48_2
@@ -11453,10 +11453,10 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v15, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v16, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v17, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v19, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v18, v5, v4, 16
; SI-NEXT: v_alignbit_b32 v21, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v24, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v7
@@ -11478,10 +11478,10 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v15, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v16, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v17, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v19, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v18, v5, v4, 16
; SI-NEXT: v_alignbit_b32 v21, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v24, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v7
@@ -11494,14 +11494,14 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v0, v0, v24
@@ -11510,9 +11510,9 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v2, v2, v21
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v26
-; SI-NEXT: v_or_b32_e32 v4, v4, v19
+; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
; SI-NEXT: v_or_b32_e32 v6, v6, v17
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v23
@@ -11524,10 +11524,10 @@ define <28 x i16> @bitcast_v7f64_to_v28i16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v20
; SI-NEXT: v_or_b32_e32 v12, v12, v14
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v18
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v19
; SI-NEXT: v_or_b32_e32 v1, v1, v24
; SI-NEXT: v_or_b32_e32 v3, v3, v21
-; SI-NEXT: v_or_b32_e32 v5, v5, v19
+; SI-NEXT: v_or_b32_e32 v5, v5, v18
; SI-NEXT: v_or_b32_e32 v7, v7, v17
; SI-NEXT: v_or_b32_e32 v9, v9, v16
; SI-NEXT: v_or_b32_e32 v11, v11, v15
@@ -12609,7 +12609,7 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr26
-; SI-NEXT: ; implicit-def: $vgpr19
+; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr23
@@ -12618,7 +12618,7 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr15
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr14
-; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB52_2
@@ -12627,10 +12627,10 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v15, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v16, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v17, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v19, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v18, v5, v4, 16
; SI-NEXT: v_alignbit_b32 v21, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v24, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v7
@@ -12652,10 +12652,10 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v15, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v16, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v17, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v19, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v18, v5, v4, 16
; SI-NEXT: v_alignbit_b32 v21, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v24, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v7
@@ -12668,14 +12668,14 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v0, v0, v24
@@ -12684,9 +12684,9 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v2, v2, v21
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v26
-; SI-NEXT: v_or_b32_e32 v4, v4, v19
+; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
; SI-NEXT: v_or_b32_e32 v6, v6, v17
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v23
@@ -12698,10 +12698,10 @@ define <28 x half> @bitcast_v7f64_to_v28f16(<7 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v20
; SI-NEXT: v_or_b32_e32 v12, v12, v14
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v18
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v19
; SI-NEXT: v_or_b32_e32 v1, v1, v24
; SI-NEXT: v_or_b32_e32 v3, v3, v21
-; SI-NEXT: v_or_b32_e32 v5, v5, v19
+; SI-NEXT: v_or_b32_e32 v5, v5, v18
; SI-NEXT: v_or_b32_e32 v7, v7, v17
; SI-NEXT: v_or_b32_e32 v9, v9, v16
; SI-NEXT: v_or_b32_e32 v11, v11, v15
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
index f79e89b17eee4..d355d882c3ea5 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
@@ -11604,42 +11604,42 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v34, v6
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; SI-NEXT: v_mov_b32_e32 v33, v4
; SI-NEXT: v_mov_b32_e32 v32, v2
; SI-NEXT: v_mov_b32_e32 v31, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:24
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:32
; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:40
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:96
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:112
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:120
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:128
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:128
; SI-NEXT: v_lshlrev_b32_e32 v22, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
; SI-NEXT: v_lshlrev_b32_e32 v28, 24, v27
+; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
; SI-NEXT: v_lshlrev_b32_e32 v30, 8, v29
-; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v1
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v3
-; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v5
+; SI-NEXT: v_lshlrev_b32_e32 v48, 8, v1
+; SI-NEXT: v_lshlrev_b32_e32 v39, 24, v3
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v5
; SI-NEXT: v_lshlrev_b32_e32 v35, 24, v7
; SI-NEXT: v_lshlrev_b32_e32 v36, 8, v9
; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v11
@@ -11660,21 +11660,21 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v14
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v49
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v51
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v50
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v53
+; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v52
; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v55
+; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v53
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v40
+; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v42
+; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v40
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v44
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:36
@@ -11684,47 +11684,43 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:116
+; SI-NEXT: s_waitcnt vmcnt(9)
+; SI-NEXT: v_lshlrev_b32_e32 v23, 8, v44
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v58
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v47
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:60
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB26_2
; SI-NEXT: ; %bb.1: ; %cmp.false
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
; SI-NEXT: v_and_b32_e32 v1, 0xff, v32
; SI-NEXT: v_and_b32_e32 v2, 0xff, v33
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v34
-; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_or_b32_e32 v0, v0, v48
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v2, v2, v39
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
-; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v34
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v3
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v9, 0xff, v57
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v62, v9
@@ -11743,12 +11739,9 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v53
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr52
@@ -11756,32 +11749,26 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr62
; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr27
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v2, v2, v36
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v2, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v1, v2, v3
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
@@ -11790,18 +11777,34 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
+; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
-; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v36
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_or_b32_e32 v3, v37, v3
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
@@ -11811,14 +11814,16 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
@@ -11868,7 +11873,7 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v13, v13, v19
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v13, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xff, v23
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v49
; SI-NEXT: v_or_b32_e32 v14, v14, v25
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_or_b32_e32 v14, v14, v15
@@ -11879,7 +11884,7 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr61
; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr40
-; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; implicit-def: $vgpr59
@@ -11890,13 +11895,8 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v15, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v15, v23
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
@@ -11923,43 +11923,40 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; kill: killed $vgpr16
+; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: ; kill: killed $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: .LBB26_2: ; %Flow
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB26_4
; SI-NEXT: ; %bb.3: ; %cmp.true
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v31
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v32
-; SI-NEXT: v_or_b32_e32 v0, v49, v0
+; SI-NEXT: v_or_b32_e32 v0, v48, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v33
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v34
-; SI-NEXT: v_or_b32_e32 v1, v39, v1
+; SI-NEXT: v_or_b32_e32 v1, v34, v1
; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v35, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v57
; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
@@ -11984,50 +11981,58 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x3000000, v0
-; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v2, v35, v2
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
-; SI-NEXT: v_or_b32_e32 v2, v36, v2
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v16, v4
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v5
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v6, vcc, 3, v6
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v36, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v3, v2
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v7, vcc, 3, v7
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
@@ -12039,14 +12044,16 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v3
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v18, v4
@@ -12116,7 +12123,7 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v13, vcc, s6, v13
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v23
+; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v49
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_or_b32_e32 v14, v25, v14
; SI-NEXT: v_add_i32_e32 v14, vcc, s6, v14
@@ -12134,15 +12141,9 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v15
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v23, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x300, v15
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v15
; SI-NEXT: .LBB26_4: ; %end
@@ -12188,8 +12189,8 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: v_mov_b32_e32 v53, v1
@@ -12251,10 +12252,10 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:104
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:100
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
@@ -12340,8 +12341,8 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr59
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v6, v7, v6, s6
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v14, v15, v14, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -12370,8 +12371,8 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v58, v24, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; VI-NEXT: ; implicit-def: $vgpr24
; VI-NEXT: ; kill: killed $vgpr24
; VI-NEXT: ; implicit-def: $vgpr24
@@ -12503,7 +12504,7 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b16_e32 v12, 8, v42
; VI-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; VI-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
@@ -12512,7 +12513,7 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; VI-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: v_add_u16_e32 v6, 0x300, v6
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v8, 8, v8
@@ -12591,8 +12592,8 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_sdwa v14, v16, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v14, v14, v15 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(1)
; VI-NEXT: v_add_u16_e32 v14, 3, v14
; VI-NEXT: s_waitcnt vmcnt(0)
@@ -12664,11 +12665,13 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
; GFX9-NEXT: v_mov_b32_e32 v53, v1
; GFX9-NEXT: v_mov_b32_e32 v54, v0
; GFX9-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:128
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:80
@@ -12684,12 +12687,12 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:16
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
; GFX9-NEXT: v_mov_b32_e32 v31, v15
; GFX9-NEXT: v_mov_b32_e32 v32, v14
@@ -12705,9 +12708,9 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_mov_b32_e32 v50, v4
; GFX9-NEXT: v_mov_b32_e32 v51, v3
; GFX9-NEXT: v_mov_b32_e32 v52, v2
-; GFX9-NEXT: s_waitcnt vmcnt(24)
+; GFX9-NEXT: s_waitcnt vmcnt(25)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; GFX9-NEXT: s_waitcnt vmcnt(22)
+; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:124
; GFX9-NEXT: s_nop 0
@@ -12731,15 +12734,13 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:92
-; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; GFX9-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; GFX9-NEXT: s_and_saveexec_b64 s[4:5], vcc
; GFX9-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB26_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s6, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v0, v54, v53, s6
; GFX9-NEXT: v_perm_b32 v1, v52, v51, s6
@@ -12757,14 +12758,8 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v5, v18, v19, s6
; GFX9-NEXT: v_lshl_or_b32 v4, v5, 16, v4
; GFX9-NEXT: v_perm_b32 v5, v20, v21, s6
-; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v9, v47, v46, s6
-; GFX9-NEXT: v_perm_b32 v10, v45, v44, s6
+; GFX9-NEXT: v_perm_b32 v9, v25, v46, s6
+; GFX9-NEXT: v_perm_b32 v10, v27, v44, s6
; GFX9-NEXT: v_perm_b32 v11, v42, v30, s6
; GFX9-NEXT: v_perm_b32 v12, v55, v63, s6
; GFX9-NEXT: v_perm_b32 v13, v60, v59, s6
@@ -12789,11 +12784,9 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr19
; GFX9-NEXT: ; implicit-def: $vgpr20
; GFX9-NEXT: ; implicit-def: $vgpr21
-; GFX9-NEXT: ; implicit-def: $vgpr22
-; GFX9-NEXT: ; implicit-def: $vgpr23
-; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr44
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr30
@@ -12801,12 +12794,34 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr60
; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_perm_b32 v6, v22, v6, s6
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_perm_b32 v14, v23, v14, s6
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s6
; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_perm_b32 v14, v15, v14, s6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s6
; GFX9-NEXT: v_lshl_or_b32 v6, v7, 16, v6
@@ -12818,9 +12833,9 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v8, v8, v43, s6
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
-; GFX9-NEXT: v_perm_b32 v8, v24, v25, s6
+; GFX9-NEXT: v_perm_b32 v8, v24, v47, s6
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
-; GFX9-NEXT: v_perm_b32 v9, v26, v27, s6
+; GFX9-NEXT: v_perm_b32 v9, v26, v45, s6
; GFX9-NEXT: v_lshl_or_b32 v9, v10, 16, v9
; GFX9-NEXT: v_perm_b32 v10, v28, v29, s6
; GFX9-NEXT: v_lshl_or_b32 v10, v11, 16, v10
@@ -12834,23 +12849,11 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-NEXT: ; implicit-def: $vgpr24
; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
; GFX9-NEXT: ; implicit-def: $vgpr43
; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr25
+; GFX9-NEXT: ; implicit-def: $vgpr47
; GFX9-NEXT: ; implicit-def: $vgpr26
-; GFX9-NEXT: ; implicit-def: $vgpr27
+; GFX9-NEXT: ; implicit-def: $vgpr45
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
; GFX9-NEXT: ; implicit-def: $vgpr41
@@ -12888,8 +12891,6 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; kill: killed $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; kill: killed $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: v_lshl_or_b32 v15, v16, 16, v15
; GFX9-NEXT: ; implicit-def: $vgpr16
; GFX9-NEXT: ; kill: killed $vgpr56
@@ -12898,6 +12899,7 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB26_4
; GFX9-NEXT: ; %bb.3: ; %cmp.true
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v0, 3, v54
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v53
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -12947,36 +12949,39 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_lshlrev_b16_e32 v6, 8, v21
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v6, 3, v22
-; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v23
-; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v43
; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v46
; GFX9-NEXT: v_lshlrev_b16_e32 v11, 8, v44
; GFX9-NEXT: v_lshlrev_b16_e32 v12, 8, v30
; GFX9-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; GFX9-NEXT: s_waitcnt vmcnt(5)
-; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: s_waitcnt vmcnt(1)
-; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(5)
; GFX9-NEXT: v_lshlrev_b16_e32 v16, 8, v16
+; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v17, 8, v17
+; GFX9-NEXT: s_waitcnt vmcnt(3)
+; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
+; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
+; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v7, 3, v7
; GFX9-NEXT: v_or_b32_sdwa v7, v8, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -12997,17 +13002,17 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
; GFX9-NEXT: v_add_u16_e32 v8, 3, v24
-; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v25
+; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v47
; GFX9-NEXT: v_or_b32_sdwa v8, v9, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v9, 3, v47
+; GFX9-NEXT: v_add_u16_e32 v9, 3, v25
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
; GFX9-NEXT: v_add_u16_e32 v9, 3, v26
-; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v27
+; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v45
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v10, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v10, 3, v27
; GFX9-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v10
@@ -13039,11 +13044,9 @@ define <16 x i32> @bitcast_v64i8_to_v16i32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v13, 3, v58
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v57
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_add_u16_e32 v14, 3, v14
+; GFX9-NEXT: v_add_u16_e32 v14, 3, v23
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v14
; GFX9-NEXT: v_lshl_or_b32 v13, v14, 16, v13
; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
@@ -14301,19 +14304,20 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s42, v18
; VI-NEXT: v_readfirstlane_b32 s43, v17
; VI-NEXT: v_readfirstlane_b32 s44, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32
+; VI-NEXT: v_readfirstlane_b32 s56, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
-; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:28
@@ -14324,7 +14328,6 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s45, v15
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s47, v13
-; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
; VI-NEXT: v_readfirstlane_b32 s58, v10
; VI-NEXT: v_readfirstlane_b32 s59, v9
@@ -14338,7 +14341,7 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s75, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB27_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -14410,8 +14413,8 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_or_b32_e32 v10, v10, v12
-; VI-NEXT: v_perm_b32 v12, v24, v22, s4
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
+; VI-NEXT: v_perm_b32 v12, v25, v24, s4
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_e32 v11, v11, v12
; VI-NEXT: s_waitcnt vmcnt(2)
@@ -14420,13 +14423,13 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v34, v33, s4
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_perm_b32 v13, v28, v27, s4
+; VI-NEXT: v_perm_b32 v13, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v13, v14, v13
; VI-NEXT: v_perm_b32 v14, v21, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; VI-NEXT: v_perm_b32 v15, v25, v23, s4
+; VI-NEXT: v_perm_b32 v15, v23, v22, s4
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_perm_b32 v15, v17, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -14545,9 +14548,9 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_or_b32_sdwa v10, v12, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
-; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v24
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
-; VI-NEXT: v_perm_b32 v12, v12, v22, s4
+; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v25
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
+; VI-NEXT: v_perm_b32 v12, v12, v24, s4
; VI-NEXT: v_add_u32_e32 v11, vcc, 0x300, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -14562,13 +14565,13 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v30
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v28
; VI-NEXT: v_perm_b32 v13, v13, v29, s4
-; VI-NEXT: v_perm_b32 v14, v14, v27, s4
+; VI-NEXT: v_perm_b32 v14, v14, v26, s4
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v25
+; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v23
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v21
-; VI-NEXT: v_perm_b32 v14, v14, v23, s4
+; VI-NEXT: v_perm_b32 v14, v14, v22, s4
; VI-NEXT: v_perm_b32 v15, v15, v20, s4
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -14618,18 +14621,19 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:8
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:4
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32
+; GFX9-NEXT: v_readfirstlane_b32 s56, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
@@ -14641,7 +14645,6 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
; GFX9-NEXT: v_readfirstlane_b32 s47, v13
-; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
; GFX9-NEXT: v_readfirstlane_b32 s58, v10
; GFX9-NEXT: v_readfirstlane_b32 s59, v9
@@ -14655,7 +14658,7 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB27_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -14728,9 +14731,9 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_or_b32_e32 v10, v10, v12
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_perm_b32 v12, v23, v21, s4
+; GFX9-NEXT: v_perm_b32 v12, v24, v22, s4
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; GFX9-NEXT: v_or_b32_e32 v11, v11, v12
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -14743,9 +14746,9 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; GFX9-NEXT: v_perm_b32 v14, v30, v29, s4
; GFX9-NEXT: v_or_b32_e32 v13, v14, v13
-; GFX9-NEXT: v_perm_b32 v14, v22, v20, s4
+; GFX9-NEXT: v_perm_b32 v14, v21, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v26, v24, s4
+; GFX9-NEXT: v_perm_b32 v15, v25, v23, s4
; GFX9-NEXT: v_or_b32_e32 v14, v15, v14
; GFX9-NEXT: v_perm_b32 v15, v17, v16, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -14866,10 +14869,10 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: s_add_i32 s6, s6, 3
; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_add_u32_e32 v12, 3, v23
+; GFX9-NEXT: v_add_u32_e32 v12, 3, v24
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
-; GFX9-NEXT: v_perm_b32 v12, v12, v21, s5
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
+; GFX9-NEXT: v_perm_b32 v12, v12, v22, s5
; GFX9-NEXT: v_add_u32_e32 v11, 0x300, v11
; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v11, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -14888,9 +14891,9 @@ define inreg <16 x i32> @bitcast_v64i8_to_v16i32_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v14, 3, v26
-; GFX9-NEXT: v_add_u32_e32 v15, 3, v22
-; GFX9-NEXT: v_perm_b32 v14, v14, v24, s5
+; GFX9-NEXT: v_add_u32_e32 v14, 3, v25
+; GFX9-NEXT: v_add_u32_e32 v15, 3, v21
+; GFX9-NEXT: v_perm_b32 v14, v14, v23, s5
; GFX9-NEXT: v_perm_b32 v15, v15, v20, s5
; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -17281,14 +17284,14 @@ define inreg <32 x i16> @bitcast_v16f32_to_v32i16_scalar(<16 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[22:23], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[23:24], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v1
; SI-NEXT: s_branch .LBB37_6
; SI-NEXT: .LBB37_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -17307,14 +17310,14 @@ define inreg <32 x i16> @bitcast_v16f32_to_v32i16_scalar(<16 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v13, s29
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_mov_b32_e32 v32, s56
-; SI-NEXT: v_mov_b32_e32 v31, s57
-; SI-NEXT: v_mov_b32_e32 v30, s58
-; SI-NEXT: v_mov_b32_e32 v29, s59
-; SI-NEXT: v_mov_b32_e32 v28, s60
-; SI-NEXT: v_mov_b32_e32 v27, s61
-; SI-NEXT: v_mov_b32_e32 v26, s62
-; SI-NEXT: v_mov_b32_e32 v25, s63
+; SI-NEXT: v_mov_b32_e32 v31, s56
+; SI-NEXT: v_mov_b32_e32 v30, s57
+; SI-NEXT: v_mov_b32_e32 v29, s58
+; SI-NEXT: v_mov_b32_e32 v28, s59
+; SI-NEXT: v_mov_b32_e32 v27, s60
+; SI-NEXT: v_mov_b32_e32 v26, s61
+; SI-NEXT: v_mov_b32_e32 v25, s62
+; SI-NEXT: v_mov_b32_e32 v24, s63
; SI-NEXT: v_mov_b32_e32 v16, s6
; SI-NEXT: v_mov_b32_e32 v17, s8
; SI-NEXT: v_mov_b32_e32 v18, s10
@@ -17334,36 +17337,36 @@ define inreg <32 x i16> @bitcast_v16f32_to_v32i16_scalar(<16 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v0, v0, v23
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v31
; SI-NEXT: v_or_b32_e32 v2, v2, v22
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v30
; SI-NEXT: v_or_b32_e32 v4, v4, v21
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v6, v6, v20
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v28
; SI-NEXT: v_or_b32_e32 v8, v8, v19
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v27
; SI-NEXT: v_or_b32_e32 v10, v10, v18
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v27
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v26
; SI-NEXT: v_or_b32_e32 v12, v12, v17
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v25
; SI-NEXT: v_or_b32_e32 v14, v14, v16
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v24
; SI-NEXT: v_or_b32_e32 v1, v1, v23
; SI-NEXT: v_or_b32_e32 v3, v3, v22
; SI-NEXT: v_or_b32_e32 v5, v5, v21
@@ -18663,14 +18666,14 @@ define inreg <32 x half> @bitcast_v16f32_to_v32f16_scalar(<16 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[22:23], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[23:24], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v1
; SI-NEXT: s_branch .LBB41_6
; SI-NEXT: .LBB41_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -18689,14 +18692,14 @@ define inreg <32 x half> @bitcast_v16f32_to_v32f16_scalar(<16 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v13, s29
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_mov_b32_e32 v32, s56
-; SI-NEXT: v_mov_b32_e32 v31, s57
-; SI-NEXT: v_mov_b32_e32 v30, s58
-; SI-NEXT: v_mov_b32_e32 v29, s59
-; SI-NEXT: v_mov_b32_e32 v28, s60
-; SI-NEXT: v_mov_b32_e32 v27, s61
-; SI-NEXT: v_mov_b32_e32 v26, s62
-; SI-NEXT: v_mov_b32_e32 v25, s63
+; SI-NEXT: v_mov_b32_e32 v31, s56
+; SI-NEXT: v_mov_b32_e32 v30, s57
+; SI-NEXT: v_mov_b32_e32 v29, s58
+; SI-NEXT: v_mov_b32_e32 v28, s59
+; SI-NEXT: v_mov_b32_e32 v27, s60
+; SI-NEXT: v_mov_b32_e32 v26, s61
+; SI-NEXT: v_mov_b32_e32 v25, s62
+; SI-NEXT: v_mov_b32_e32 v24, s63
; SI-NEXT: v_mov_b32_e32 v16, s6
; SI-NEXT: v_mov_b32_e32 v17, s8
; SI-NEXT: v_mov_b32_e32 v18, s10
@@ -18716,36 +18719,36 @@ define inreg <32 x half> @bitcast_v16f32_to_v32f16_scalar(<16 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v0, v0, v23
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v31
; SI-NEXT: v_or_b32_e32 v2, v2, v22
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v30
; SI-NEXT: v_or_b32_e32 v4, v4, v21
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v6, v6, v20
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v28
; SI-NEXT: v_or_b32_e32 v8, v8, v19
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v27
; SI-NEXT: v_or_b32_e32 v10, v10, v18
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v27
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v26
; SI-NEXT: v_or_b32_e32 v12, v12, v17
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v25
; SI-NEXT: v_or_b32_e32 v14, v14, v16
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v24
; SI-NEXT: v_or_b32_e32 v1, v1, v23
; SI-NEXT: v_or_b32_e32 v3, v3, v22
; SI-NEXT: v_or_b32_e32 v5, v5, v21
@@ -20263,6 +20266,9 @@ define inreg <32 x bfloat> @bitcast_v16f32_to_v32bf16_scalar(<16 x float> inreg
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB45_5
; SI-NEXT: ; %bb.4: ; %cmp.true
+; SI-NEXT: v_add_f32_e64 v26, s18, 1.0
+; SI-NEXT: v_add_f32_e64 v24, s19, 1.0
+; SI-NEXT: v_add_f32_e64 v22, s20, 1.0
; SI-NEXT: v_add_f32_e64 v20, s21, 1.0
; SI-NEXT: v_add_f32_e64 v18, s22, 1.0
; SI-NEXT: v_add_f32_e64 v16, s23, 1.0
@@ -20274,9 +20280,6 @@ define inreg <32 x bfloat> @bitcast_v16f32_to_v32bf16_scalar(<16 x float> inreg
; SI-NEXT: v_add_f32_e64 v4, s29, 1.0
; SI-NEXT: v_add_f32_e64 v2, s7, 1.0
; SI-NEXT: v_add_f32_e64 v0, s6, 1.0
-; SI-NEXT: v_add_f32_e64 v22, s20, 1.0
-; SI-NEXT: v_add_f32_e64 v24, s19, 1.0
-; SI-NEXT: v_add_f32_e64 v26, s18, 1.0
; SI-NEXT: v_add_f32_e64 v28, s17, 1.0
; SI-NEXT: v_add_f32_e64 v30, s16, 1.0
; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v0
@@ -26823,42 +26826,42 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v34, v6
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; SI-NEXT: v_mov_b32_e32 v33, v4
; SI-NEXT: v_mov_b32_e32 v32, v2
; SI-NEXT: v_mov_b32_e32 v31, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:24
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:32
; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:40
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:96
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:112
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:120
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:128
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:128
; SI-NEXT: v_lshlrev_b32_e32 v22, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
; SI-NEXT: v_lshlrev_b32_e32 v28, 24, v27
+; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
; SI-NEXT: v_lshlrev_b32_e32 v30, 8, v29
-; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v1
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v3
-; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v5
+; SI-NEXT: v_lshlrev_b32_e32 v48, 8, v1
+; SI-NEXT: v_lshlrev_b32_e32 v39, 24, v3
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v5
; SI-NEXT: v_lshlrev_b32_e32 v35, 24, v7
; SI-NEXT: v_lshlrev_b32_e32 v36, 8, v9
; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v11
@@ -26879,21 +26882,21 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v14
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v49
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v51
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v50
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v53
+; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v52
; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v55
+; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v53
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v40
+; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v42
+; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v40
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v44
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:36
@@ -26903,47 +26906,43 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:116
+; SI-NEXT: s_waitcnt vmcnt(9)
+; SI-NEXT: v_lshlrev_b32_e32 v23, 8, v44
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v58
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v47
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:60
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB50_2
; SI-NEXT: ; %bb.1: ; %cmp.false
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
; SI-NEXT: v_and_b32_e32 v1, 0xff, v32
; SI-NEXT: v_and_b32_e32 v2, 0xff, v33
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v34
-; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_or_b32_e32 v0, v0, v48
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v2, v2, v39
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
-; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v34
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v3
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v9, 0xff, v57
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v62, v9
@@ -26962,12 +26961,9 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v53
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr52
@@ -26975,32 +26971,26 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr62
; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr27
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v2, v2, v36
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v2, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v1, v2, v3
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
@@ -27009,18 +26999,34 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
+; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
-; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v36
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_or_b32_e32 v3, v37, v3
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
@@ -27030,14 +27036,16 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
@@ -27087,7 +27095,7 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v13, v13, v19
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v13, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xff, v23
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v49
; SI-NEXT: v_or_b32_e32 v14, v14, v25
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_or_b32_e32 v14, v14, v15
@@ -27098,7 +27106,7 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr61
; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr40
-; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; implicit-def: $vgpr59
@@ -27109,13 +27117,8 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v15, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v15, v23
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
@@ -27142,43 +27145,40 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; kill: killed $vgpr16
+; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: ; kill: killed $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: .LBB50_2: ; %Flow
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB50_4
; SI-NEXT: ; %bb.3: ; %cmp.true
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v31
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v32
-; SI-NEXT: v_or_b32_e32 v0, v49, v0
+; SI-NEXT: v_or_b32_e32 v0, v48, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v33
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v34
-; SI-NEXT: v_or_b32_e32 v1, v39, v1
+; SI-NEXT: v_or_b32_e32 v1, v34, v1
; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v35, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v57
; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
@@ -27203,50 +27203,58 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x3000000, v0
-; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v2, v35, v2
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
-; SI-NEXT: v_or_b32_e32 v2, v36, v2
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v16, v4
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v5
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v6, vcc, 3, v6
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v36, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v3, v2
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v7, vcc, 3, v7
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
@@ -27258,14 +27266,16 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v3
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v18, v4
@@ -27335,7 +27345,7 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v13, vcc, s6, v13
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v23
+; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v49
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_or_b32_e32 v14, v25, v14
; SI-NEXT: v_add_i32_e32 v14, vcc, s6, v14
@@ -27353,15 +27363,9 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v15
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v23, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x300, v15
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v15
; SI-NEXT: .LBB50_4: ; %end
@@ -27407,8 +27411,8 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: v_mov_b32_e32 v53, v1
@@ -27470,10 +27474,10 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:104
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:100
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
@@ -27559,8 +27563,8 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr59
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v6, v7, v6, s6
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v14, v15, v14, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -27589,8 +27593,8 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v58, v24, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; VI-NEXT: ; implicit-def: $vgpr24
; VI-NEXT: ; kill: killed $vgpr24
; VI-NEXT: ; implicit-def: $vgpr24
@@ -27722,7 +27726,7 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b16_e32 v12, 8, v42
; VI-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; VI-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
@@ -27731,7 +27735,7 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; VI-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: v_add_u16_e32 v6, 0x300, v6
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v8, 8, v8
@@ -27810,8 +27814,8 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_sdwa v14, v16, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v14, v14, v15 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(1)
; VI-NEXT: v_add_u16_e32 v14, 3, v14
; VI-NEXT: s_waitcnt vmcnt(0)
@@ -27883,11 +27887,13 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
; GFX9-NEXT: v_mov_b32_e32 v53, v1
; GFX9-NEXT: v_mov_b32_e32 v54, v0
; GFX9-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:128
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:80
@@ -27903,12 +27909,12 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:16
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
; GFX9-NEXT: v_mov_b32_e32 v31, v15
; GFX9-NEXT: v_mov_b32_e32 v32, v14
@@ -27924,9 +27930,9 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_mov_b32_e32 v50, v4
; GFX9-NEXT: v_mov_b32_e32 v51, v3
; GFX9-NEXT: v_mov_b32_e32 v52, v2
-; GFX9-NEXT: s_waitcnt vmcnt(24)
+; GFX9-NEXT: s_waitcnt vmcnt(25)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; GFX9-NEXT: s_waitcnt vmcnt(22)
+; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:124
; GFX9-NEXT: s_nop 0
@@ -27950,15 +27956,13 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:92
-; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; GFX9-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; GFX9-NEXT: s_and_saveexec_b64 s[4:5], vcc
; GFX9-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB50_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s6, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v0, v54, v53, s6
; GFX9-NEXT: v_perm_b32 v1, v52, v51, s6
@@ -27976,14 +27980,8 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v5, v18, v19, s6
; GFX9-NEXT: v_lshl_or_b32 v4, v5, 16, v4
; GFX9-NEXT: v_perm_b32 v5, v20, v21, s6
-; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v9, v47, v46, s6
-; GFX9-NEXT: v_perm_b32 v10, v45, v44, s6
+; GFX9-NEXT: v_perm_b32 v9, v25, v46, s6
+; GFX9-NEXT: v_perm_b32 v10, v27, v44, s6
; GFX9-NEXT: v_perm_b32 v11, v42, v30, s6
; GFX9-NEXT: v_perm_b32 v12, v55, v63, s6
; GFX9-NEXT: v_perm_b32 v13, v60, v59, s6
@@ -28008,11 +28006,9 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr19
; GFX9-NEXT: ; implicit-def: $vgpr20
; GFX9-NEXT: ; implicit-def: $vgpr21
-; GFX9-NEXT: ; implicit-def: $vgpr22
-; GFX9-NEXT: ; implicit-def: $vgpr23
-; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr44
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr30
@@ -28020,12 +28016,34 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr60
; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_perm_b32 v6, v22, v6, s6
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_perm_b32 v14, v23, v14, s6
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s6
; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_perm_b32 v14, v15, v14, s6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s6
; GFX9-NEXT: v_lshl_or_b32 v6, v7, 16, v6
@@ -28037,9 +28055,9 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v8, v8, v43, s6
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
-; GFX9-NEXT: v_perm_b32 v8, v24, v25, s6
+; GFX9-NEXT: v_perm_b32 v8, v24, v47, s6
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
-; GFX9-NEXT: v_perm_b32 v9, v26, v27, s6
+; GFX9-NEXT: v_perm_b32 v9, v26, v45, s6
; GFX9-NEXT: v_lshl_or_b32 v9, v10, 16, v9
; GFX9-NEXT: v_perm_b32 v10, v28, v29, s6
; GFX9-NEXT: v_lshl_or_b32 v10, v11, 16, v10
@@ -28053,23 +28071,11 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-NEXT: ; implicit-def: $vgpr24
; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
; GFX9-NEXT: ; implicit-def: $vgpr43
; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr25
+; GFX9-NEXT: ; implicit-def: $vgpr47
; GFX9-NEXT: ; implicit-def: $vgpr26
-; GFX9-NEXT: ; implicit-def: $vgpr27
+; GFX9-NEXT: ; implicit-def: $vgpr45
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
; GFX9-NEXT: ; implicit-def: $vgpr41
@@ -28107,8 +28113,6 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; kill: killed $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; kill: killed $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: v_lshl_or_b32 v15, v16, 16, v15
; GFX9-NEXT: ; implicit-def: $vgpr16
; GFX9-NEXT: ; kill: killed $vgpr56
@@ -28117,6 +28121,7 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB50_4
; GFX9-NEXT: ; %bb.3: ; %cmp.true
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v0, 3, v54
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v53
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -28166,36 +28171,39 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_lshlrev_b16_e32 v6, 8, v21
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v6, 3, v22
-; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v23
-; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v43
; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v46
; GFX9-NEXT: v_lshlrev_b16_e32 v11, 8, v44
; GFX9-NEXT: v_lshlrev_b16_e32 v12, 8, v30
; GFX9-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; GFX9-NEXT: s_waitcnt vmcnt(5)
-; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: s_waitcnt vmcnt(1)
-; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(5)
; GFX9-NEXT: v_lshlrev_b16_e32 v16, 8, v16
+; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v17, 8, v17
+; GFX9-NEXT: s_waitcnt vmcnt(3)
+; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
+; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
+; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v7, 3, v7
; GFX9-NEXT: v_or_b32_sdwa v7, v8, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -28216,17 +28224,17 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
; GFX9-NEXT: v_add_u16_e32 v8, 3, v24
-; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v25
+; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v47
; GFX9-NEXT: v_or_b32_sdwa v8, v9, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v9, 3, v47
+; GFX9-NEXT: v_add_u16_e32 v9, 3, v25
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
; GFX9-NEXT: v_add_u16_e32 v9, 3, v26
-; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v27
+; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v45
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v10, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v10, 3, v27
; GFX9-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v10
@@ -28258,11 +28266,9 @@ define <16 x float> @bitcast_v64i8_to_v16f32(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v13, 3, v58
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v57
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_add_u16_e32 v14, 3, v14
+; GFX9-NEXT: v_add_u16_e32 v14, 3, v23
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v14
; GFX9-NEXT: v_lshl_or_b32 v13, v14, 16, v13
; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
@@ -29520,19 +29526,20 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s42, v18
; VI-NEXT: v_readfirstlane_b32 s43, v17
; VI-NEXT: v_readfirstlane_b32 s44, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32
+; VI-NEXT: v_readfirstlane_b32 s56, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
-; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:28
@@ -29543,7 +29550,6 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s45, v15
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s47, v13
-; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
; VI-NEXT: v_readfirstlane_b32 s58, v10
; VI-NEXT: v_readfirstlane_b32 s59, v9
@@ -29557,7 +29563,7 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: v_readfirstlane_b32 s75, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -29629,8 +29635,8 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_or_b32_e32 v10, v10, v12
-; VI-NEXT: v_perm_b32 v12, v24, v22, s4
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
+; VI-NEXT: v_perm_b32 v12, v25, v24, s4
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_e32 v11, v11, v12
; VI-NEXT: s_waitcnt vmcnt(2)
@@ -29639,13 +29645,13 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v34, v33, s4
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_perm_b32 v13, v28, v27, s4
+; VI-NEXT: v_perm_b32 v13, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v13, v14, v13
; VI-NEXT: v_perm_b32 v14, v21, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; VI-NEXT: v_perm_b32 v15, v25, v23, s4
+; VI-NEXT: v_perm_b32 v15, v23, v22, s4
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_perm_b32 v15, v17, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -29764,9 +29770,9 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: v_or_b32_sdwa v10, v12, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
-; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v24
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
-; VI-NEXT: v_perm_b32 v12, v12, v22, s4
+; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v25
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
+; VI-NEXT: v_perm_b32 v12, v12, v24, s4
; VI-NEXT: v_add_u32_e32 v11, vcc, 0x300, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -29781,13 +29787,13 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v30
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v28
; VI-NEXT: v_perm_b32 v13, v13, v29, s4
-; VI-NEXT: v_perm_b32 v14, v14, v27, s4
+; VI-NEXT: v_perm_b32 v14, v14, v26, s4
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v25
+; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v23
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v21
-; VI-NEXT: v_perm_b32 v14, v14, v23, s4
+; VI-NEXT: v_perm_b32 v14, v14, v22, s4
; VI-NEXT: v_perm_b32 v15, v15, v20, s4
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -29837,18 +29843,19 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:8
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:4
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32
+; GFX9-NEXT: v_readfirstlane_b32 s56, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
@@ -29860,7 +29867,6 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
; GFX9-NEXT: v_readfirstlane_b32 s47, v13
-; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
; GFX9-NEXT: v_readfirstlane_b32 s58, v10
; GFX9-NEXT: v_readfirstlane_b32 s59, v9
@@ -29874,7 +29880,7 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB51_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -29947,9 +29953,9 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_or_b32_e32 v10, v10, v12
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_perm_b32 v12, v23, v21, s4
+; GFX9-NEXT: v_perm_b32 v12, v24, v22, s4
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; GFX9-NEXT: v_or_b32_e32 v11, v11, v12
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -29962,9 +29968,9 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; GFX9-NEXT: v_perm_b32 v14, v30, v29, s4
; GFX9-NEXT: v_or_b32_e32 v13, v14, v13
-; GFX9-NEXT: v_perm_b32 v14, v22, v20, s4
+; GFX9-NEXT: v_perm_b32 v14, v21, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v26, v24, s4
+; GFX9-NEXT: v_perm_b32 v15, v25, v23, s4
; GFX9-NEXT: v_or_b32_e32 v14, v15, v14
; GFX9-NEXT: v_perm_b32 v15, v17, v16, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -30085,10 +30091,10 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: s_add_i32 s6, s6, 3
; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_add_u32_e32 v12, 3, v23
+; GFX9-NEXT: v_add_u32_e32 v12, 3, v24
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
-; GFX9-NEXT: v_perm_b32 v12, v12, v21, s5
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
+; GFX9-NEXT: v_perm_b32 v12, v12, v22, s5
; GFX9-NEXT: v_add_u32_e32 v11, 0x300, v11
; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v11, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -30107,9 +30113,9 @@ define inreg <16 x float> @bitcast_v64i8_to_v16f32_scalar(<64 x i8> inreg %a, i3
; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v14, 3, v26
-; GFX9-NEXT: v_add_u32_e32 v15, 3, v22
-; GFX9-NEXT: v_perm_b32 v14, v14, v24, s5
+; GFX9-NEXT: v_add_u32_e32 v14, 3, v25
+; GFX9-NEXT: v_add_u32_e32 v15, 3, v21
+; GFX9-NEXT: v_perm_b32 v14, v14, v23, s5
; GFX9-NEXT: v_perm_b32 v15, v15, v20, s5
; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -40941,42 +40947,42 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v34, v6
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; SI-NEXT: v_mov_b32_e32 v33, v4
; SI-NEXT: v_mov_b32_e32 v32, v2
; SI-NEXT: v_mov_b32_e32 v31, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:24
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:32
; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:40
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:96
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:112
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:120
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:128
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:128
; SI-NEXT: v_lshlrev_b32_e32 v22, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
; SI-NEXT: v_lshlrev_b32_e32 v28, 24, v27
+; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
; SI-NEXT: v_lshlrev_b32_e32 v30, 8, v29
-; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v1
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v3
-; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v5
+; SI-NEXT: v_lshlrev_b32_e32 v48, 8, v1
+; SI-NEXT: v_lshlrev_b32_e32 v39, 24, v3
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v5
; SI-NEXT: v_lshlrev_b32_e32 v35, 24, v7
; SI-NEXT: v_lshlrev_b32_e32 v36, 8, v9
; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v11
@@ -40997,21 +41003,21 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v14
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v49
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v51
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v50
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v53
+; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v52
; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v55
+; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v53
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v40
+; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v42
+; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v40
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v44
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:36
@@ -41021,47 +41027,43 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:116
+; SI-NEXT: s_waitcnt vmcnt(9)
+; SI-NEXT: v_lshlrev_b32_e32 v23, 8, v44
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v58
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v47
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:60
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB70_2
; SI-NEXT: ; %bb.1: ; %cmp.false
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
; SI-NEXT: v_and_b32_e32 v1, 0xff, v32
; SI-NEXT: v_and_b32_e32 v2, 0xff, v33
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v34
-; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_or_b32_e32 v0, v0, v48
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v2, v2, v39
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
-; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v34
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v3
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v9, 0xff, v57
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v62, v9
@@ -41080,12 +41082,9 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v53
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr52
@@ -41093,32 +41092,26 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr62
; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr27
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v2, v2, v36
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v2, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v1, v2, v3
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
@@ -41127,18 +41120,34 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
+; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
-; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v36
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_or_b32_e32 v3, v37, v3
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
@@ -41148,14 +41157,16 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
@@ -41205,7 +41216,7 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v13, v13, v19
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v13, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xff, v23
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v49
; SI-NEXT: v_or_b32_e32 v14, v14, v25
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_or_b32_e32 v14, v14, v15
@@ -41216,7 +41227,7 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr61
; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr40
-; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; implicit-def: $vgpr59
@@ -41227,13 +41238,8 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v15, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v15, v23
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
@@ -41260,43 +41266,40 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; kill: killed $vgpr16
+; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: ; kill: killed $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: .LBB70_2: ; %Flow
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB70_4
; SI-NEXT: ; %bb.3: ; %cmp.true
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v31
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v32
-; SI-NEXT: v_or_b32_e32 v0, v49, v0
+; SI-NEXT: v_or_b32_e32 v0, v48, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v33
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v34
-; SI-NEXT: v_or_b32_e32 v1, v39, v1
+; SI-NEXT: v_or_b32_e32 v1, v34, v1
; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v35, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v57
; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
@@ -41321,50 +41324,58 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x3000000, v0
-; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v2, v35, v2
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
-; SI-NEXT: v_or_b32_e32 v2, v36, v2
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v16, v4
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v5
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v6, vcc, 3, v6
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v36, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v3, v2
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v7, vcc, 3, v7
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
@@ -41376,14 +41387,16 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v3
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v18, v4
@@ -41453,7 +41466,7 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v13, vcc, s6, v13
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v23
+; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v49
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_or_b32_e32 v14, v25, v14
; SI-NEXT: v_add_i32_e32 v14, vcc, s6, v14
@@ -41471,15 +41484,9 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v15
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v23, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x300, v15
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v15
; SI-NEXT: .LBB70_4: ; %end
@@ -41525,8 +41532,8 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: v_mov_b32_e32 v53, v1
@@ -41588,10 +41595,10 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:104
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:100
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
@@ -41677,8 +41684,8 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr59
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v6, v7, v6, s6
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v14, v15, v14, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -41707,8 +41714,8 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v58, v24, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; VI-NEXT: ; implicit-def: $vgpr24
; VI-NEXT: ; kill: killed $vgpr24
; VI-NEXT: ; implicit-def: $vgpr24
@@ -41840,7 +41847,7 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b16_e32 v12, 8, v42
; VI-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; VI-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
@@ -41849,7 +41856,7 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; VI-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: v_add_u16_e32 v6, 0x300, v6
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v8, 8, v8
@@ -41928,8 +41935,8 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_sdwa v14, v16, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v14, v14, v15 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(1)
; VI-NEXT: v_add_u16_e32 v14, 3, v14
; VI-NEXT: s_waitcnt vmcnt(0)
@@ -42001,11 +42008,13 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
; GFX9-NEXT: v_mov_b32_e32 v53, v1
; GFX9-NEXT: v_mov_b32_e32 v54, v0
; GFX9-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:128
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:80
@@ -42021,12 +42030,12 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:16
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
; GFX9-NEXT: v_mov_b32_e32 v31, v15
; GFX9-NEXT: v_mov_b32_e32 v32, v14
@@ -42042,9 +42051,9 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_mov_b32_e32 v50, v4
; GFX9-NEXT: v_mov_b32_e32 v51, v3
; GFX9-NEXT: v_mov_b32_e32 v52, v2
-; GFX9-NEXT: s_waitcnt vmcnt(24)
+; GFX9-NEXT: s_waitcnt vmcnt(25)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; GFX9-NEXT: s_waitcnt vmcnt(22)
+; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:124
; GFX9-NEXT: s_nop 0
@@ -42068,15 +42077,13 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:92
-; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; GFX9-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; GFX9-NEXT: s_and_saveexec_b64 s[4:5], vcc
; GFX9-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB70_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s6, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v0, v54, v53, s6
; GFX9-NEXT: v_perm_b32 v1, v52, v51, s6
@@ -42094,14 +42101,8 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v5, v18, v19, s6
; GFX9-NEXT: v_lshl_or_b32 v4, v5, 16, v4
; GFX9-NEXT: v_perm_b32 v5, v20, v21, s6
-; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v9, v47, v46, s6
-; GFX9-NEXT: v_perm_b32 v10, v45, v44, s6
+; GFX9-NEXT: v_perm_b32 v9, v25, v46, s6
+; GFX9-NEXT: v_perm_b32 v10, v27, v44, s6
; GFX9-NEXT: v_perm_b32 v11, v42, v30, s6
; GFX9-NEXT: v_perm_b32 v12, v55, v63, s6
; GFX9-NEXT: v_perm_b32 v13, v60, v59, s6
@@ -42126,11 +42127,9 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr19
; GFX9-NEXT: ; implicit-def: $vgpr20
; GFX9-NEXT: ; implicit-def: $vgpr21
-; GFX9-NEXT: ; implicit-def: $vgpr22
-; GFX9-NEXT: ; implicit-def: $vgpr23
-; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr44
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr30
@@ -42138,12 +42137,34 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr60
; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_perm_b32 v6, v22, v6, s6
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_perm_b32 v14, v23, v14, s6
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s6
; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_perm_b32 v14, v15, v14, s6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s6
; GFX9-NEXT: v_lshl_or_b32 v6, v7, 16, v6
@@ -42155,9 +42176,9 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v8, v8, v43, s6
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
-; GFX9-NEXT: v_perm_b32 v8, v24, v25, s6
+; GFX9-NEXT: v_perm_b32 v8, v24, v47, s6
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
-; GFX9-NEXT: v_perm_b32 v9, v26, v27, s6
+; GFX9-NEXT: v_perm_b32 v9, v26, v45, s6
; GFX9-NEXT: v_lshl_or_b32 v9, v10, 16, v9
; GFX9-NEXT: v_perm_b32 v10, v28, v29, s6
; GFX9-NEXT: v_lshl_or_b32 v10, v11, 16, v10
@@ -42171,23 +42192,11 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-NEXT: ; implicit-def: $vgpr24
; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
; GFX9-NEXT: ; implicit-def: $vgpr43
; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr25
+; GFX9-NEXT: ; implicit-def: $vgpr47
; GFX9-NEXT: ; implicit-def: $vgpr26
-; GFX9-NEXT: ; implicit-def: $vgpr27
+; GFX9-NEXT: ; implicit-def: $vgpr45
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
; GFX9-NEXT: ; implicit-def: $vgpr41
@@ -42225,8 +42234,6 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; kill: killed $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; kill: killed $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: v_lshl_or_b32 v15, v16, 16, v15
; GFX9-NEXT: ; implicit-def: $vgpr16
; GFX9-NEXT: ; kill: killed $vgpr56
@@ -42235,6 +42242,7 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB70_4
; GFX9-NEXT: ; %bb.3: ; %cmp.true
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v0, 3, v54
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v53
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -42284,36 +42292,39 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_lshlrev_b16_e32 v6, 8, v21
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v6, 3, v22
-; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v23
-; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v43
; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v46
; GFX9-NEXT: v_lshlrev_b16_e32 v11, 8, v44
; GFX9-NEXT: v_lshlrev_b16_e32 v12, 8, v30
; GFX9-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; GFX9-NEXT: s_waitcnt vmcnt(5)
-; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: s_waitcnt vmcnt(1)
-; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(5)
; GFX9-NEXT: v_lshlrev_b16_e32 v16, 8, v16
+; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v17, 8, v17
+; GFX9-NEXT: s_waitcnt vmcnt(3)
+; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
+; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
+; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v7, 3, v7
; GFX9-NEXT: v_or_b32_sdwa v7, v8, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -42334,17 +42345,17 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
; GFX9-NEXT: v_add_u16_e32 v8, 3, v24
-; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v25
+; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v47
; GFX9-NEXT: v_or_b32_sdwa v8, v9, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v9, 3, v47
+; GFX9-NEXT: v_add_u16_e32 v9, 3, v25
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
; GFX9-NEXT: v_add_u16_e32 v9, 3, v26
-; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v27
+; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v45
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v10, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v10, 3, v27
; GFX9-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v10
@@ -42376,11 +42387,9 @@ define <8 x i64> @bitcast_v64i8_to_v8i64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v13, 3, v58
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v57
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_add_u16_e32 v14, 3, v14
+; GFX9-NEXT: v_add_u16_e32 v14, 3, v23
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v14
; GFX9-NEXT: v_lshl_or_b32 v13, v14, 16, v13
; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
@@ -43638,19 +43647,20 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: v_readfirstlane_b32 s42, v18
; VI-NEXT: v_readfirstlane_b32 s43, v17
; VI-NEXT: v_readfirstlane_b32 s44, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32
+; VI-NEXT: v_readfirstlane_b32 s56, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
-; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:28
@@ -43661,7 +43671,6 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: v_readfirstlane_b32 s45, v15
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s47, v13
-; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
; VI-NEXT: v_readfirstlane_b32 s58, v10
; VI-NEXT: v_readfirstlane_b32 s59, v9
@@ -43675,7 +43684,7 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: v_readfirstlane_b32 s75, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB71_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -43747,8 +43756,8 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_or_b32_e32 v10, v10, v12
-; VI-NEXT: v_perm_b32 v12, v24, v22, s4
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
+; VI-NEXT: v_perm_b32 v12, v25, v24, s4
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_e32 v11, v11, v12
; VI-NEXT: s_waitcnt vmcnt(2)
@@ -43757,13 +43766,13 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v34, v33, s4
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_perm_b32 v13, v28, v27, s4
+; VI-NEXT: v_perm_b32 v13, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v13, v14, v13
; VI-NEXT: v_perm_b32 v14, v21, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; VI-NEXT: v_perm_b32 v15, v25, v23, s4
+; VI-NEXT: v_perm_b32 v15, v23, v22, s4
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_perm_b32 v15, v17, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -43882,9 +43891,9 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: v_or_b32_sdwa v10, v12, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
-; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v24
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
-; VI-NEXT: v_perm_b32 v12, v12, v22, s4
+; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v25
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
+; VI-NEXT: v_perm_b32 v12, v12, v24, s4
; VI-NEXT: v_add_u32_e32 v11, vcc, 0x300, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -43899,13 +43908,13 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v30
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v28
; VI-NEXT: v_perm_b32 v13, v13, v29, s4
-; VI-NEXT: v_perm_b32 v14, v14, v27, s4
+; VI-NEXT: v_perm_b32 v14, v14, v26, s4
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v25
+; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v23
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v21
-; VI-NEXT: v_perm_b32 v14, v14, v23, s4
+; VI-NEXT: v_perm_b32 v14, v14, v22, s4
; VI-NEXT: v_perm_b32 v15, v15, v20, s4
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -43955,18 +43964,19 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:8
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:4
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32
+; GFX9-NEXT: v_readfirstlane_b32 s56, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
@@ -43978,7 +43988,6 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
; GFX9-NEXT: v_readfirstlane_b32 s47, v13
-; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
; GFX9-NEXT: v_readfirstlane_b32 s58, v10
; GFX9-NEXT: v_readfirstlane_b32 s59, v9
@@ -43992,7 +44001,7 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB71_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -44065,9 +44074,9 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_or_b32_e32 v10, v10, v12
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_perm_b32 v12, v23, v21, s4
+; GFX9-NEXT: v_perm_b32 v12, v24, v22, s4
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; GFX9-NEXT: v_or_b32_e32 v11, v11, v12
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -44080,9 +44089,9 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; GFX9-NEXT: v_perm_b32 v14, v30, v29, s4
; GFX9-NEXT: v_or_b32_e32 v13, v14, v13
-; GFX9-NEXT: v_perm_b32 v14, v22, v20, s4
+; GFX9-NEXT: v_perm_b32 v14, v21, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v26, v24, s4
+; GFX9-NEXT: v_perm_b32 v15, v25, v23, s4
; GFX9-NEXT: v_or_b32_e32 v14, v15, v14
; GFX9-NEXT: v_perm_b32 v15, v17, v16, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -44203,10 +44212,10 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: s_add_i32 s6, s6, 3
; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_add_u32_e32 v12, 3, v23
+; GFX9-NEXT: v_add_u32_e32 v12, 3, v24
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
-; GFX9-NEXT: v_perm_b32 v12, v12, v21, s5
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
+; GFX9-NEXT: v_perm_b32 v12, v12, v22, s5
; GFX9-NEXT: v_add_u32_e32 v11, 0x300, v11
; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v11, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -44225,9 +44234,9 @@ define inreg <8 x i64> @bitcast_v64i8_to_v8i64_scalar(<64 x i8> inreg %a, i32 in
; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v14, 3, v26
-; GFX9-NEXT: v_add_u32_e32 v15, 3, v22
-; GFX9-NEXT: v_perm_b32 v14, v14, v24, s5
+; GFX9-NEXT: v_add_u32_e32 v14, 3, v25
+; GFX9-NEXT: v_add_u32_e32 v15, 3, v21
+; GFX9-NEXT: v_perm_b32 v14, v14, v23, s5
; GFX9-NEXT: v_perm_b32 v15, v15, v20, s5
; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -44842,22 +44851,22 @@ define <32 x i16> @bitcast_v8f64_to_v32i16(<8 x double> %a, i32 %b) #0 {
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v16
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr22
+; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr20
-; SI-NEXT: ; implicit-def: $vgpr27
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr18
-; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr21
+; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB72_2
@@ -44867,14 +44876,14 @@ define <32 x i16> @bitcast_v8f64_to_v32i16(<8 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v18, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v19, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v20, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v22, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v25, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v28, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v15
+; SI-NEXT: v_alignbit_b32 v21, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v24, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v27, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v3
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v1
@@ -44895,62 +44904,62 @@ define <32 x i16> @bitcast_v8f64_to_v32i16(<8 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v18, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v19, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v20, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v22, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v25, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v28, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v15
+; SI-NEXT: v_alignbit_b32 v21, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v24, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v27, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v3
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v1
; SI-NEXT: .LBB72_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v0, v0, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v27
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v31
-; SI-NEXT: v_or_b32_e32 v2, v2, v25
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v31
+; SI-NEXT: v_or_b32_e32 v2, v2, v24
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v30
-; SI-NEXT: v_or_b32_e32 v4, v4, v22
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v30
+; SI-NEXT: v_or_b32_e32 v4, v4, v21
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v6, v6, v20
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v28
; SI-NEXT: v_or_b32_e32 v8, v8, v19
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v26
; SI-NEXT: v_or_b32_e32 v10, v10, v18
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
; SI-NEXT: v_or_b32_e32 v12, v12, v17
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v23
; SI-NEXT: v_or_b32_e32 v14, v14, v16
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v21
-; SI-NEXT: v_or_b32_e32 v1, v1, v28
-; SI-NEXT: v_or_b32_e32 v3, v3, v25
-; SI-NEXT: v_or_b32_e32 v5, v5, v22
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v22
+; SI-NEXT: v_or_b32_e32 v1, v1, v27
+; SI-NEXT: v_or_b32_e32 v3, v3, v24
+; SI-NEXT: v_or_b32_e32 v5, v5, v21
; SI-NEXT: v_or_b32_e32 v7, v7, v20
; SI-NEXT: v_or_b32_e32 v9, v9, v19
; SI-NEXT: v_or_b32_e32 v11, v11, v18
@@ -46168,22 +46177,22 @@ define <32 x half> @bitcast_v8f64_to_v32f16(<8 x double> %a, i32 %b) #0 {
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v16
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr22
+; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr20
-; SI-NEXT: ; implicit-def: $vgpr27
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr18
-; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr21
+; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB76_2
@@ -46193,14 +46202,14 @@ define <32 x half> @bitcast_v8f64_to_v32f16(<8 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v18, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v19, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v20, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v22, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v25, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v28, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v15
+; SI-NEXT: v_alignbit_b32 v21, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v24, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v27, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v3
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v1
@@ -46221,62 +46230,62 @@ define <32 x half> @bitcast_v8f64_to_v32f16(<8 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v18, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v19, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v20, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v22, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v25, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v28, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v15
+; SI-NEXT: v_alignbit_b32 v21, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v24, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v27, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v3
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v1
; SI-NEXT: .LBB76_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v0, v0, v28
+; SI-NEXT: v_or_b32_e32 v0, v0, v27
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v31
-; SI-NEXT: v_or_b32_e32 v2, v2, v25
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v31
+; SI-NEXT: v_or_b32_e32 v2, v2, v24
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v30
-; SI-NEXT: v_or_b32_e32 v4, v4, v22
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v30
+; SI-NEXT: v_or_b32_e32 v4, v4, v21
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v6, v6, v20
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v28
; SI-NEXT: v_or_b32_e32 v8, v8, v19
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v26
; SI-NEXT: v_or_b32_e32 v10, v10, v18
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v25
; SI-NEXT: v_or_b32_e32 v12, v12, v17
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v23
; SI-NEXT: v_or_b32_e32 v14, v14, v16
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v21
-; SI-NEXT: v_or_b32_e32 v1, v1, v28
-; SI-NEXT: v_or_b32_e32 v3, v3, v25
-; SI-NEXT: v_or_b32_e32 v5, v5, v22
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v22
+; SI-NEXT: v_or_b32_e32 v1, v1, v27
+; SI-NEXT: v_or_b32_e32 v3, v3, v24
+; SI-NEXT: v_or_b32_e32 v5, v5, v21
; SI-NEXT: v_or_b32_e32 v7, v7, v20
; SI-NEXT: v_or_b32_e32 v9, v9, v19
; SI-NEXT: v_or_b32_e32 v11, v11, v18
@@ -47976,13 +47985,13 @@ define inreg <32 x bfloat> @bitcast_v8f64_to_v32bf16_scalar(<8 x double> inreg %
; SI-NEXT: s_cmp_lg_u32 s6, 1
; SI-NEXT: s_cbranch_scc1 .LBB81_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_add_f64 v[19:20], s[22:23], 1.0
+; SI-NEXT: v_add_f64 v[26:27], s[18:19], 1.0
+; SI-NEXT: v_add_f64 v[22:23], s[20:21], 1.0
+; SI-NEXT: v_add_f64 v[18:19], s[22:23], 1.0
; SI-NEXT: v_add_f64 v[14:15], s[24:25], 1.0
; SI-NEXT: v_add_f64 v[10:11], s[26:27], 1.0
; SI-NEXT: v_add_f64 v[2:3], s[4:5], 1.0
; SI-NEXT: v_add_f64 v[6:7], s[28:29], 1.0
-; SI-NEXT: v_add_f64 v[23:24], s[20:21], 1.0
-; SI-NEXT: v_add_f64 v[27:28], s[18:19], 1.0
; SI-NEXT: v_add_f64 v[30:31], s[16:17], 1.0
; SI-NEXT: v_and_b32_e32 v1, 0xffff0000, v3
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v3
@@ -48000,18 +48009,18 @@ define inreg <32 x bfloat> @bitcast_v8f64_to_v32bf16_scalar(<8 x double> inreg %
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v15
; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v14
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v20
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v20
-; SI-NEXT: v_and_b32_e32 v18, 0xffff0000, v19
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
-; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v24
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v24
-; SI-NEXT: v_and_b32_e32 v22, 0xffff0000, v23
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
-; SI-NEXT: v_and_b32_e32 v25, 0xffff0000, v28
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v28
-; SI-NEXT: v_and_b32_e32 v26, 0xffff0000, v27
-; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_and_b32_e32 v17, 0xffff0000, v19
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v19
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v18
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v23
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v23
+; SI-NEXT: v_and_b32_e32 v23, 0xffff0000, v22
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_and_b32_e32 v25, 0xffff0000, v27
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v27
+; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v26
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; SI-NEXT: v_and_b32_e32 v29, 0xffff0000, v31
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v31
; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v30
@@ -48036,16 +48045,16 @@ define inreg <32 x bfloat> @bitcast_v8f64_to_v32bf16_scalar(<8 x double> inreg %
; SI-NEXT: v_mov_b32_e32 v14, s56
; SI-NEXT: v_mov_b32_e32 v17, s47
; SI-NEXT: v_mov_b32_e32 v16, s46
-; SI-NEXT: v_mov_b32_e32 v18, s45
-; SI-NEXT: v_mov_b32_e32 v19, s44
+; SI-NEXT: v_mov_b32_e32 v19, s45
+; SI-NEXT: v_mov_b32_e32 v18, s44
; SI-NEXT: v_mov_b32_e32 v21, s43
; SI-NEXT: v_mov_b32_e32 v20, s42
-; SI-NEXT: v_mov_b32_e32 v22, s41
-; SI-NEXT: v_mov_b32_e32 v23, s40
+; SI-NEXT: v_mov_b32_e32 v23, s41
+; SI-NEXT: v_mov_b32_e32 v22, s40
; SI-NEXT: v_mov_b32_e32 v25, s15
; SI-NEXT: v_mov_b32_e32 v24, s14
-; SI-NEXT: v_mov_b32_e32 v26, s13
-; SI-NEXT: v_mov_b32_e32 v27, s12
+; SI-NEXT: v_mov_b32_e32 v27, s13
+; SI-NEXT: v_mov_b32_e32 v26, s12
; SI-NEXT: v_mov_b32_e32 v29, s11
; SI-NEXT: v_mov_b32_e32 v28, s10
; SI-NEXT: v_mov_b32_e32 v31, s9
@@ -48058,34 +48067,34 @@ define inreg <32 x bfloat> @bitcast_v8f64_to_v32bf16_scalar(<8 x double> inreg %
; SI-NEXT: v_mul_f32_e32 v1, 1.0, v1
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v29
; SI-NEXT: v_mul_f32_e32 v28, 1.0, v28
-; SI-NEXT: v_mul_f32_e32 v26, 1.0, v26
+; SI-NEXT: v_mul_f32_e32 v27, 1.0, v27
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v1
; SI-NEXT: v_mul_f32_e32 v50, 1.0, v0
; SI-NEXT: v_lshr_b64 v[0:1], v[30:31], 16
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v26
-; SI-NEXT: v_mul_f32_e32 v32, 1.0, v27
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_mul_f32_e32 v26, 1.0, v26
; SI-NEXT: v_mul_f32_e32 v25, 1.0, v25
; SI-NEXT: v_mul_f32_e32 v3, 1.0, v3
; SI-NEXT: v_mul_f32_e32 v48, 1.0, v2
; SI-NEXT: v_lshr_b64 v[1:2], v[28:29], 16
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v25
; SI-NEXT: v_mul_f32_e32 v24, 1.0, v24
-; SI-NEXT: v_mul_f32_e32 v22, 1.0, v22
+; SI-NEXT: v_mul_f32_e32 v23, 1.0, v23
; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v3
-; SI-NEXT: v_lshr_b64 v[2:3], v[32:33], 16
-; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v22
-; SI-NEXT: v_mul_f32_e32 v26, 1.0, v23
+; SI-NEXT: v_lshr_b64 v[2:3], v[26:27], 16
+; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_mul_f32_e32 v22, 1.0, v22
; SI-NEXT: v_mul_f32_e32 v21, 1.0, v21
; SI-NEXT: v_mul_f32_e32 v5, 1.0, v5
; SI-NEXT: v_mul_f32_e32 v38, 1.0, v4
; SI-NEXT: v_lshr_b64 v[3:4], v[24:25], 16
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v21
; SI-NEXT: v_mul_f32_e32 v20, 1.0, v20
-; SI-NEXT: v_mul_f32_e32 v18, 1.0, v18
+; SI-NEXT: v_mul_f32_e32 v19, 1.0, v19
; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v5
-; SI-NEXT: v_lshr_b64 v[4:5], v[26:27], 16
-; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v18
-; SI-NEXT: v_mul_f32_e32 v22, 1.0, v19
+; SI-NEXT: v_lshr_b64 v[4:5], v[22:23], 16
+; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_mul_f32_e32 v18, 1.0, v18
; SI-NEXT: v_mul_f32_e32 v17, 1.0, v17
; SI-NEXT: v_mul_f32_e32 v7, 1.0, v7
; SI-NEXT: v_mul_f32_e32 v36, 1.0, v6
@@ -48094,7 +48103,7 @@ define inreg <32 x bfloat> @bitcast_v8f64_to_v32bf16_scalar(<8 x double> inreg %
; SI-NEXT: v_mul_f32_e32 v16, 1.0, v16
; SI-NEXT: v_mul_f32_e32 v15, 1.0, v15
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v7
-; SI-NEXT: v_lshr_b64 v[6:7], v[22:23], 16
+; SI-NEXT: v_lshr_b64 v[6:7], v[18:19], 16
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v15
; SI-NEXT: v_mul_f32_e32 v14, 1.0, v14
; SI-NEXT: v_mul_f32_e32 v13, 1.0, v13
@@ -48106,10 +48115,10 @@ define inreg <32 x bfloat> @bitcast_v8f64_to_v32bf16_scalar(<8 x double> inreg %
; SI-NEXT: v_mul_f32_e32 v11, 1.0, v11
; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
; SI-NEXT: v_lshr_b64 v[8:9], v[14:15], 16
-; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v11
-; SI-NEXT: v_mul_f32_e32 v18, 1.0, v10
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v11
+; SI-NEXT: v_mul_f32_e32 v32, 1.0, v10
; SI-NEXT: v_lshr_b64 v[9:10], v[12:13], 16
-; SI-NEXT: v_lshr_b64 v[10:11], v[18:19], 16
+; SI-NEXT: v_lshr_b64 v[10:11], v[32:33], 16
; SI-NEXT: v_lshr_b64 v[11:12], v[34:35], 16
; SI-NEXT: v_lshr_b64 v[12:13], v[36:37], 16
; SI-NEXT: v_lshr_b64 v[13:14], v[38:39], 16
@@ -54478,42 +54487,42 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_store_dword v20, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; SI-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v34, v6
+; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; SI-NEXT: v_mov_b32_e32 v33, v4
; SI-NEXT: v_mov_b32_e32 v32, v2
; SI-NEXT: v_mov_b32_e32 v31, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:24
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:32
; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:40
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:96
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:112
-; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:120
-; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:128
-; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:120
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:128
; SI-NEXT: v_lshlrev_b32_e32 v22, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
; SI-NEXT: v_lshlrev_b32_e32 v28, 24, v27
+; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v25
; SI-NEXT: v_lshlrev_b32_e32 v30, 8, v29
-; SI-NEXT: v_lshlrev_b32_e32 v49, 8, v1
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v3
-; SI-NEXT: v_lshlrev_b32_e32 v39, 8, v5
+; SI-NEXT: v_lshlrev_b32_e32 v48, 8, v1
+; SI-NEXT: v_lshlrev_b32_e32 v39, 24, v3
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v5
; SI-NEXT: v_lshlrev_b32_e32 v35, 24, v7
; SI-NEXT: v_lshlrev_b32_e32 v36, 8, v9
; SI-NEXT: v_lshlrev_b32_e32 v37, 24, v11
@@ -54534,21 +54543,21 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v14
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v50
+; SI-NEXT: v_lshlrev_b32_e32 v41, 8, v49
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v51
+; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v50
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v52
+; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v51
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v53
+; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v52
; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v55
+; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v53
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v40
+; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v42
+; SI-NEXT: v_lshlrev_b32_e32 v25, 8, v40
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v44
+; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v42
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:36
@@ -54558,47 +54567,43 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v60, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:116
+; SI-NEXT: s_waitcnt vmcnt(9)
+; SI-NEXT: v_lshlrev_b32_e32 v23, 8, v44
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v58
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v47
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v58, off, s[0:3], s32 offset:60
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:124
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB86_2
; SI-NEXT: ; %bb.1: ; %cmp.false
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xff, v31
; SI-NEXT: v_and_b32_e32 v1, 0xff, v32
; SI-NEXT: v_and_b32_e32 v2, 0xff, v33
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v34
-; SI-NEXT: v_or_b32_e32 v0, v0, v49
+; SI-NEXT: v_or_b32_e32 v0, v0, v48
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v2, v2, v39
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
-; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v34
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_or_b32_e32 v1, v2, v3
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_and_b32_e32 v9, 0xff, v57
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v62, v9
@@ -54617,12 +54622,9 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v53
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v29
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr52
@@ -54630,32 +54632,26 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr44
; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr62
; SI-NEXT: ; implicit-def: $vgpr56
; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr27
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v2, v2, v36
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v2, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: v_or_b32_e32 v3, v35, v3
+; SI-NEXT: v_or_b32_e32 v1, v2, v3
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
@@ -54664,18 +54660,34 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
+; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
-; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr24
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v36
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_or_b32_e32 v3, v37, v3
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
@@ -54685,14 +54697,16 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v4, v18
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
@@ -54742,7 +54756,7 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v13, v13, v19
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v13, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xff, v23
+; SI-NEXT: v_and_b32_e32 v14, 0xff, v49
; SI-NEXT: v_or_b32_e32 v14, v14, v25
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_or_b32_e32 v14, v14, v15
@@ -54753,7 +54767,7 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr61
; SI-NEXT: ; implicit-def: $vgpr47
; SI-NEXT: ; implicit-def: $vgpr40
-; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr63
; SI-NEXT: ; implicit-def: $vgpr59
@@ -54764,13 +54778,8 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v15, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v15, v23
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
@@ -54797,43 +54806,40 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr16
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; kill: killed $vgpr16
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; kill: killed $vgpr50
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; kill: killed $vgpr16
+; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: ; kill: killed $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: .LBB86_2: ; %Flow
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB86_4
; SI-NEXT: ; %bb.3: ; %cmp.true
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v31
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v32
-; SI-NEXT: v_or_b32_e32 v0, v49, v0
+; SI-NEXT: v_or_b32_e32 v0, v48, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x300, v0
-; SI-NEXT: v_or_b32_e32 v1, v48, v1
+; SI-NEXT: v_or_b32_e32 v1, v39, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v33
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v34
-; SI-NEXT: v_or_b32_e32 v1, v39, v1
+; SI-NEXT: v_or_b32_e32 v1, v34, v1
; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v35, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v57
; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
@@ -54858,50 +54864,58 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; SI-NEXT: v_or_b32_e32 v14, v27, v14
-; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v29
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x3000000, v0
-; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v2, v35, v2
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
-; SI-NEXT: v_or_b32_e32 v2, v36, v2
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
; SI-NEXT: v_or_b32_e32 v3, v37, v3
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v2, v3, v2
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v16, v4
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v5
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v20, v5
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v6, vcc, 3, v6
; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v24, v6
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v36, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v2, v3, v2
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v7, vcc, 3, v7
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v28, v7
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
@@ -54913,14 +54927,16 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v3
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
+; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
; SI-NEXT: v_or_b32_e32 v8, v17, v8
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v16, v50, v16
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
; SI-NEXT: v_or_b32_e32 v4, v18, v4
@@ -54990,7 +55006,7 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v13, vcc, s6, v13
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v23
+; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v49
; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
; SI-NEXT: v_or_b32_e32 v14, v25, v14
; SI-NEXT: v_add_i32_e32 v14, vcc, s6, v14
@@ -55008,15 +55024,9 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v15
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
-; SI-NEXT: v_or_b32_e32 v15, v16, v15
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v15, v23, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x300, v15
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v16, vcc, 3, v16
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
-; SI-NEXT: v_or_b32_e32 v16, v50, v16
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v15
; SI-NEXT: .LBB86_4: ; %end
@@ -55062,8 +55072,8 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v30, off, s[0:3], s32 offset:208 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:216 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v28, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
-; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: v_mov_b32_e32 v53, v1
@@ -55125,10 +55135,10 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:104
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:100
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
@@ -55214,8 +55224,8 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr59
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v6, v7, v6, s6
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_perm_b32 v14, v15, v14, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -55244,8 +55254,8 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v12, v12, v13
; VI-NEXT: v_perm_b32 v13, v58, v24, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; VI-NEXT: ; implicit-def: $vgpr24
; VI-NEXT: ; kill: killed $vgpr24
; VI-NEXT: ; implicit-def: $vgpr24
@@ -55377,7 +55387,7 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b16_e32 v12, 8, v42
; VI-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; VI-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; VI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
@@ -55386,7 +55396,7 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; VI-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; VI-NEXT: v_add_u16_e32 v6, 0x300, v6
; VI-NEXT: s_waitcnt vmcnt(4)
; VI-NEXT: v_lshlrev_b16_e32 v8, 8, v8
@@ -55465,8 +55475,8 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_sdwa v14, v16, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v14, v14, v15 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v13, v13, v14
-; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; VI-NEXT: s_waitcnt vmcnt(1)
; VI-NEXT: v_add_u16_e32 v14, 3, v14
; VI-NEXT: s_waitcnt vmcnt(0)
@@ -55538,11 +55548,13 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v26, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v25, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v24, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v23, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
; GFX9-NEXT: v_mov_b32_e32 v53, v1
; GFX9-NEXT: v_mov_b32_e32 v54, v0
; GFX9-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:132
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:128
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:92
; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:80
@@ -55558,12 +55570,12 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:24
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:20
; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:16
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
; GFX9-NEXT: v_mov_b32_e32 v31, v15
; GFX9-NEXT: v_mov_b32_e32 v32, v14
@@ -55579,9 +55591,9 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_mov_b32_e32 v50, v4
; GFX9-NEXT: v_mov_b32_e32 v51, v3
; GFX9-NEXT: v_mov_b32_e32 v52, v2
-; GFX9-NEXT: s_waitcnt vmcnt(24)
+; GFX9-NEXT: s_waitcnt vmcnt(25)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; GFX9-NEXT: s_waitcnt vmcnt(22)
+; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:204 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:124
; GFX9-NEXT: s_nop 0
@@ -55605,15 +55617,13 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:96
; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v1, off, s[0:3], s32 offset:92
-; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; GFX9-NEXT: ; implicit-def: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
; GFX9-NEXT: s_and_saveexec_b64 s[4:5], vcc
; GFX9-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB86_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: s_mov_b32 s6, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v0, v54, v53, s6
; GFX9-NEXT: v_perm_b32 v1, v52, v51, s6
@@ -55631,14 +55641,8 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v5, v18, v19, s6
; GFX9-NEXT: v_lshl_or_b32 v4, v5, 16, v4
; GFX9-NEXT: v_perm_b32 v5, v20, v21, s6
-; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_perm_b32 v9, v47, v46, s6
-; GFX9-NEXT: v_perm_b32 v10, v45, v44, s6
+; GFX9-NEXT: v_perm_b32 v9, v25, v46, s6
+; GFX9-NEXT: v_perm_b32 v10, v27, v44, s6
; GFX9-NEXT: v_perm_b32 v11, v42, v30, s6
; GFX9-NEXT: v_perm_b32 v12, v55, v63, s6
; GFX9-NEXT: v_perm_b32 v13, v60, v59, s6
@@ -55663,11 +55667,9 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr19
; GFX9-NEXT: ; implicit-def: $vgpr20
; GFX9-NEXT: ; implicit-def: $vgpr21
-; GFX9-NEXT: ; implicit-def: $vgpr22
-; GFX9-NEXT: ; implicit-def: $vgpr23
-; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr44
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr30
@@ -55675,12 +55677,34 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr60
; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_perm_b32 v6, v22, v6, s6
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_perm_b32 v14, v23, v14, s6
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: ; kill: killed $vgpr23
+; GFX9-NEXT: ; implicit-def: $vgpr23
+; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s6
; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_perm_b32 v14, v15, v14, s6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s6
; GFX9-NEXT: v_lshl_or_b32 v6, v7, 16, v6
@@ -55692,9 +55716,9 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_perm_b32 v8, v8, v43, s6
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
-; GFX9-NEXT: v_perm_b32 v8, v24, v25, s6
+; GFX9-NEXT: v_perm_b32 v8, v24, v47, s6
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
-; GFX9-NEXT: v_perm_b32 v9, v26, v27, s6
+; GFX9-NEXT: v_perm_b32 v9, v26, v45, s6
; GFX9-NEXT: v_lshl_or_b32 v9, v10, 16, v9
; GFX9-NEXT: v_perm_b32 v10, v28, v29, s6
; GFX9-NEXT: v_lshl_or_b32 v10, v11, 16, v10
@@ -55708,23 +55732,11 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-NEXT: ; implicit-def: $vgpr24
; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; kill: killed $vgpr24
; GFX9-NEXT: ; implicit-def: $vgpr43
; GFX9-NEXT: ; implicit-def: $vgpr24
-; GFX9-NEXT: ; implicit-def: $vgpr25
+; GFX9-NEXT: ; implicit-def: $vgpr47
; GFX9-NEXT: ; implicit-def: $vgpr26
-; GFX9-NEXT: ; implicit-def: $vgpr27
+; GFX9-NEXT: ; implicit-def: $vgpr45
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
; GFX9-NEXT: ; implicit-def: $vgpr41
@@ -55762,8 +55774,6 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; kill: killed $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; kill: killed $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: v_lshl_or_b32 v15, v16, 16, v15
; GFX9-NEXT: ; implicit-def: $vgpr16
; GFX9-NEXT: ; kill: killed $vgpr56
@@ -55772,6 +55782,7 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB86_4
; GFX9-NEXT: ; %bb.3: ; %cmp.true
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v0, 3, v54
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v53
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -55821,36 +55832,39 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_lshlrev_b16_e32 v6, 8, v21
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v6, 3, v22
-; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v23
-; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v5, 0x300, v5
-; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v43
; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v46
; GFX9-NEXT: v_lshlrev_b16_e32 v11, 8, v44
; GFX9-NEXT: v_lshlrev_b16_e32 v12, 8, v30
; GFX9-NEXT: v_lshlrev_b16_e32 v13, 8, v63
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v59
-; GFX9-NEXT: s_waitcnt vmcnt(5)
-; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
-; GFX9-NEXT: s_waitcnt vmcnt(1)
-; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: v_lshl_or_b32 v5, v6, 16, v5
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; GFX9-NEXT: s_waitcnt vmcnt(5)
; GFX9-NEXT: v_lshlrev_b16_e32 v16, 8, v16
+; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_lshlrev_b16_e32 v17, 8, v17
+; GFX9-NEXT: s_waitcnt vmcnt(3)
+; GFX9-NEXT: v_lshlrev_b16_e32 v8, 8, v8
+; GFX9-NEXT: s_waitcnt vmcnt(2)
+; GFX9-NEXT: v_lshlrev_b16_e32 v15, 8, v15
+; GFX9-NEXT: s_waitcnt vmcnt(1)
+; GFX9-NEXT: v_add_u16_e32 v6, 3, v6
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: v_lshlrev_b16_e32 v7, 8, v7
+; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; GFX9-NEXT: v_add_u16_e32 v6, 0x300, v6
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v7, 3, v7
; GFX9-NEXT: v_or_b32_sdwa v7, v8, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -55871,17 +55885,17 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_lshl_or_b32 v7, v8, 16, v7
; GFX9-NEXT: v_add_u16_e32 v8, 3, v24
-; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v25
+; GFX9-NEXT: v_lshlrev_b16_e32 v9, 8, v47
; GFX9-NEXT: v_or_b32_sdwa v8, v9, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v9, 3, v47
+; GFX9-NEXT: v_add_u16_e32 v9, 3, v25
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v8
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_lshl_or_b32 v8, v9, 16, v8
; GFX9-NEXT: v_add_u16_e32 v9, 3, v26
-; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v27
+; GFX9-NEXT: v_lshlrev_b16_e32 v10, 8, v45
; GFX9-NEXT: v_or_b32_sdwa v9, v10, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v10, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v10, 3, v27
; GFX9-NEXT: v_or_b32_sdwa v10, v11, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v9
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v10
@@ -55913,11 +55927,9 @@ define <8 x double> @bitcast_v64i8_to_v8f64(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v13, 3, v58
; GFX9-NEXT: v_lshlrev_b16_e32 v14, 8, v57
; GFX9-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: v_add_u16_e32 v14, 3, v14
+; GFX9-NEXT: v_add_u16_e32 v14, 3, v23
; GFX9-NEXT: v_or_b32_sdwa v14, v15, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v14
; GFX9-NEXT: v_lshl_or_b32 v13, v14, 16, v13
; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
@@ -57175,19 +57187,20 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s42, v18
; VI-NEXT: v_readfirstlane_b32 s43, v17
; VI-NEXT: v_readfirstlane_b32 s44, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
-; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32
+; VI-NEXT: v_readfirstlane_b32 s56, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; VI-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
; VI-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
-; VI-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; VI-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
; VI-NEXT: buffer_load_ushort v30, off, s[0:3], s32 offset:28
@@ -57198,7 +57211,6 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s45, v15
; VI-NEXT: v_readfirstlane_b32 s46, v14
; VI-NEXT: v_readfirstlane_b32 s47, v13
-; VI-NEXT: v_readfirstlane_b32 s56, v12
; VI-NEXT: v_readfirstlane_b32 s57, v11
; VI-NEXT: v_readfirstlane_b32 s58, v10
; VI-NEXT: v_readfirstlane_b32 s59, v9
@@ -57212,7 +57224,7 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s75, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB87_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -57284,8 +57296,8 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
; VI-NEXT: v_or_b32_e32 v10, v10, v12
-; VI-NEXT: v_perm_b32 v12, v24, v22, s4
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
+; VI-NEXT: v_perm_b32 v12, v25, v24, s4
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_e32 v11, v11, v12
; VI-NEXT: s_waitcnt vmcnt(2)
@@ -57294,13 +57306,13 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_perm_b32 v13, v34, v33, s4
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_perm_b32 v13, v28, v27, s4
+; VI-NEXT: v_perm_b32 v13, v28, v26, s4
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v30, v29, s4
; VI-NEXT: v_or_b32_e32 v13, v14, v13
; VI-NEXT: v_perm_b32 v14, v21, v20, s4
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; VI-NEXT: v_perm_b32 v15, v25, v23, s4
+; VI-NEXT: v_perm_b32 v15, v23, v22, s4
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_perm_b32 v15, v17, v16, s4
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -57419,9 +57431,9 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_or_b32_sdwa v10, v12, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; VI-NEXT: s_add_i32 s6, s6, 3
; VI-NEXT: s_mov_b32 s4, 0xc0c0004
-; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v24
-; VI-NEXT: v_perm_b32 v11, s6, v26, v11
-; VI-NEXT: v_perm_b32 v12, v12, v22, s4
+; VI-NEXT: v_add_u32_e32 v12, vcc, 3, v25
+; VI-NEXT: v_perm_b32 v11, s6, v27, v11
+; VI-NEXT: v_perm_b32 v12, v12, v24, s4
; VI-NEXT: v_add_u32_e32 v11, vcc, 0x300, v11
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; VI-NEXT: v_or_b32_sdwa v11, v12, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -57436,13 +57448,13 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_add_u32_e32 v13, vcc, 3, v30
; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v28
; VI-NEXT: v_perm_b32 v13, v13, v29, s4
-; VI-NEXT: v_perm_b32 v14, v14, v27, s4
+; VI-NEXT: v_perm_b32 v14, v14, v26, s4
; VI-NEXT: v_add_u32_e32 v13, vcc, 0x300, v13
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
; VI-NEXT: v_or_b32_sdwa v13, v14, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v25
+; VI-NEXT: v_add_u32_e32 v14, vcc, 3, v23
; VI-NEXT: v_add_u32_e32 v15, vcc, 3, v21
-; VI-NEXT: v_perm_b32 v14, v14, v23, s4
+; VI-NEXT: v_perm_b32 v14, v14, v22, s4
; VI-NEXT: v_perm_b32 v15, v15, v20, s4
; VI-NEXT: v_add_u32_e32 v14, vcc, 0x300, v14
; VI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -57492,18 +57504,19 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s42, v18
; GFX9-NEXT: v_readfirstlane_b32 s43, v17
; GFX9-NEXT: v_readfirstlane_b32 s44, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:8
-; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:4
-; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32
+; GFX9-NEXT: v_readfirstlane_b32 s56, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
+; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:72
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:64
; GFX9-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:60
; GFX9-NEXT: buffer_load_ushort v20, off, s[0:3], s32 offset:56
-; GFX9-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_load_ushort v24, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_load_ushort v21, off, s[0:3], s32 offset:52
+; GFX9-NEXT: buffer_load_ushort v23, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:44
; GFX9-NEXT: buffer_load_ushort v27, off, s[0:3], s32 offset:40
; GFX9-NEXT: buffer_load_ushort v28, off, s[0:3], s32 offset:36
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:32
@@ -57515,7 +57528,6 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s45, v15
; GFX9-NEXT: v_readfirstlane_b32 s46, v14
; GFX9-NEXT: v_readfirstlane_b32 s47, v13
-; GFX9-NEXT: v_readfirstlane_b32 s56, v12
; GFX9-NEXT: v_readfirstlane_b32 s57, v11
; GFX9-NEXT: v_readfirstlane_b32 s58, v10
; GFX9-NEXT: v_readfirstlane_b32 s59, v9
@@ -57529,7 +57541,7 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s75, v1
; GFX9-NEXT: v_readfirstlane_b32 s76, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB87_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -57602,9 +57614,9 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: s_mov_b32 s4, 0xc0c0004
; GFX9-NEXT: v_or_b32_e32 v10, v10, v12
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_perm_b32 v12, v23, v21, s4
+; GFX9-NEXT: v_perm_b32 v12, v24, v22, s4
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v12, 16, v12
; GFX9-NEXT: v_or_b32_e32 v11, v11, v12
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -57617,9 +57629,9 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; GFX9-NEXT: v_perm_b32 v14, v30, v29, s4
; GFX9-NEXT: v_or_b32_e32 v13, v14, v13
-; GFX9-NEXT: v_perm_b32 v14, v22, v20, s4
+; GFX9-NEXT: v_perm_b32 v14, v21, v20, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; GFX9-NEXT: v_perm_b32 v15, v26, v24, s4
+; GFX9-NEXT: v_perm_b32 v15, v25, v23, s4
; GFX9-NEXT: v_or_b32_e32 v14, v15, v14
; GFX9-NEXT: v_perm_b32 v15, v17, v16, s4
; GFX9-NEXT: v_lshlrev_b32_e32 v15, 16, v15
@@ -57740,10 +57752,10 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: s_add_i32 s6, s6, 3
; GFX9-NEXT: s_mov_b32 s5, 0xc0c0004
; GFX9-NEXT: s_waitcnt vmcnt(17)
-; GFX9-NEXT: v_add_u32_e32 v12, 3, v23
+; GFX9-NEXT: v_add_u32_e32 v12, 3, v24
; GFX9-NEXT: s_waitcnt vmcnt(16)
-; GFX9-NEXT: v_perm_b32 v11, s6, v25, v11
-; GFX9-NEXT: v_perm_b32 v12, v12, v21, s5
+; GFX9-NEXT: v_perm_b32 v11, s6, v26, v11
+; GFX9-NEXT: v_perm_b32 v12, v12, v22, s5
; GFX9-NEXT: v_add_u32_e32 v11, 0x300, v11
; GFX9-NEXT: v_add_u32_sdwa v12, v12, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v11, v11, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -57762,9 +57774,9 @@ define inreg <8 x double> @bitcast_v64i8_to_v8f64_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_add_u32_e32 v13, 0x300, v13
; GFX9-NEXT: v_add_u32_sdwa v14, v14, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX9-NEXT: v_or_b32_sdwa v13, v13, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX9-NEXT: v_add_u32_e32 v14, 3, v26
-; GFX9-NEXT: v_add_u32_e32 v15, 3, v22
-; GFX9-NEXT: v_perm_b32 v14, v14, v24, s5
+; GFX9-NEXT: v_add_u32_e32 v14, 3, v25
+; GFX9-NEXT: v_add_u32_e32 v15, 3, v21
+; GFX9-NEXT: v_perm_b32 v14, v14, v23, s5
; GFX9-NEXT: v_perm_b32 v15, v15, v20, s5
; GFX9-NEXT: v_add_u32_e32 v14, 0x300, v14
; GFX9-NEXT: v_add_u32_sdwa v15, v15, s4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -68133,370 +68145,365 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:24
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:16
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:96
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:16
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:80
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:120
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:128
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:112
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:40
-; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:92
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92
+; SI-NEXT: v_lshlrev_b32_e32 v1, 8, v1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v1, 8, v9
+; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v27
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v58, 8, v25
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:108
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:100
; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v3
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v15
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v11
+; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v15
; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v23
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v11
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v7
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr3
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v52, 8, v13
-; SI-NEXT: v_lshlrev_b32_e32 v55, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v57, 8, v29
-; SI-NEXT: v_lshlrev_b32_e32 v45, 24, v27
-; SI-NEXT: v_lshlrev_b32_e32 v53, 8, v17
-; SI-NEXT: v_lshlrev_b32_e32 v47, 8, v25
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v54, 8, v13
+; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v21
+; SI-NEXT: v_lshlrev_b32_e32 v53, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v59, 8, v29
+; SI-NEXT: v_lshlrev_b32_e32 v55, 8, v17
; SI-NEXT: ; kill: killed $vgpr3
; SI-NEXT: ; implicit-def: $vgpr3
; SI-NEXT: ; implicit-def: $vgpr5
-; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr7
+; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; kill: killed $vgpr3
; SI-NEXT: ; implicit-def: $vgpr3
-; SI-NEXT: ; implicit-def: $vgpr60
+; SI-NEXT: ; implicit-def: $vgpr45
; SI-NEXT: ; kill: killed $vgpr5
; SI-NEXT: ; implicit-def: $vgpr5
-; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr41
-; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; kill: killed $vgpr7
; SI-NEXT: ; implicit-def: $vgpr7
-; SI-NEXT: ; implicit-def: $vgpr39
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: ; implicit-def: $vgpr9
+; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr11
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr13
-; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr15
-; SI-NEXT: ; implicit-def: $vgpr50
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:124
+; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v2
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v1
-; SI-NEXT: v_lshlrev_b32_e32 v46, 24, v6
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v9
-; SI-NEXT: v_lshlrev_b32_e32 v61, 8, v4
-; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v18
-; SI-NEXT: v_lshlrev_b32_e32 v18, 8, v20
-; SI-NEXT: v_lshlrev_b32_e32 v58, 24, v10
-; SI-NEXT: v_lshlrev_b32_e32 v62, 8, v12
-; SI-NEXT: v_lshlrev_b32_e32 v33, 8, v28
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:12
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v14
-; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v22
-; SI-NEXT: v_lshlrev_b32_e32 v4, 24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v46, 24, v2
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v4
+; SI-NEXT: v_lshlrev_b32_e32 v57, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v60, 24, v8
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:124
+; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v12
+; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v14
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v18
+; SI-NEXT: v_lshlrev_b32_e32 v30, 24, v20
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v22
+; SI-NEXT: v_lshlrev_b32_e32 v27, 8, v28
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:60
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:12
+; SI-NEXT: v_lshlrev_b32_e32 v12, 24, v31
+; SI-NEXT: v_lshlrev_b32_e32 v29, 24, v32
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v33
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v31
-; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v32
-; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:76
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v34
-; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v35
-; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v36
-; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v37
-; SI-NEXT: ; implicit-def: $vgpr12
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v34
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v35
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:84
+; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:76
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:36
+; SI-NEXT: v_lshlrev_b32_e32 v52, 8, v36
; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr14
-; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr9
-; SI-NEXT: ; implicit-def: $vgpr22
-; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr32
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr22
+; SI-NEXT: ; implicit-def: $vgpr20
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:28
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:20
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:100
+; SI-NEXT: v_lshlrev_b32_e32 v16, 8, v10
+; SI-NEXT: ; implicit-def: $vgpr10
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB98_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v13, 0xff, v40
-; SI-NEXT: v_or_b32_e32 v13, v13, v18
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v8, v8, v33
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_or_b32_e32 v1, v1, v62
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v59, v2
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_or_b32_e32 v1, v24, v1
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v25
+; SI-NEXT: v_or_b32_e32 v11, v11, v16
+; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
+; SI-NEXT: v_or_b32_e32 v6, v12, v6
+; SI-NEXT: v_and_b32_e32 v12, 0xff, v62
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_or_b32_e32 v12, v29, v12
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v42
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_or_b32_e32 v13, v13, v26
+; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_or_b32_e32 v8, v63, v8
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: v_or_b32_e32 v11, v11, v8
+; SI-NEXT: v_alignbit_b32 v32, v11, v1, 16
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v8
+; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v6
+; SI-NEXT: ; implicit-def: $vgpr8
+; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: ; implicit-def: $vgpr59
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr63
+; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v24, 0xff, v24
; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v24, v24, v25
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_or_b32_e32 v18, v24, v15
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v42
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_or_b32_e32 v4, v4, v15
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v30
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_or_b32_e32 v21, v21, v15
-; SI-NEXT: v_or_b32_e32 v15, v8, v21
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_or_b32_e32 v8, v27, v8
-; SI-NEXT: v_or_b32_e32 v13, v13, v18
-; SI-NEXT: v_alignbit_b32 v26, v13, v4, 16
-; SI-NEXT: v_alignbit_b32 v32, v15, v8, 16
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v18
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v21
-; SI-NEXT: ; implicit-def: $vgpr42
+; SI-NEXT: v_or_b32_e32 v16, v30, v15
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v41
+; SI-NEXT: ; implicit-def: $vgpr41
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr18
-; SI-NEXT: ; implicit-def: $vgpr21
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_or_b32_e32 v13, v13, v16
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v16
+; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: v_or_b32_e32 v2, v2, v15
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v61
+; SI-NEXT: ; implicit-def: $vgpr61
+; SI-NEXT: v_or_b32_e32 v15, v15, v27
+; SI-NEXT: v_alignbit_b32 v22, v13, v2, 16
; SI-NEXT: ; implicit-def: $vgpr27
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v48, v46, v11
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v20
-; SI-NEXT: ; implicit-def: $vgpr46
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v20, v58, v11
-; SI-NEXT: v_or_b32_e32 v11, v1, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
-; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: ; implicit-def: $vgpr58
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v1, v63, v1
-; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: v_alignbit_b32 v22, v11, v1, 16
+; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
+; SI-NEXT: v_or_b32_e32 v15, v15, v6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_alignbit_b32 v20, v15, v12, 16
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; SI-NEXT: v_or_b32_e32 v29, v7, v5
+; SI-NEXT: v_or_b32_e32 v35, v7, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v23, v3, v29
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v21, v3, v35
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v29
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_or_b32_e32 v16, v16, v24
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v35
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v12, v7, v5
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v10, v7, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v52
+; SI-NEXT: v_or_b32_e32 v5, v5, v54
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v36, v7, v3
-; SI-NEXT: v_alignbit_b32 v3, v23, v12, 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v3, v21, v10, 16
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v12, v16, v12
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v10, v24, v10
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v3, v3, v36
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v36
-; SI-NEXT: ; implicit-def: $vgpr52
+; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v14, v7, v5
+; SI-NEXT: v_or_b32_e32 v17, v7, v5
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v5, v3, v14, 16
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v24, 0xff, v24
+; SI-NEXT: v_or_b32_e32 v24, v24, v25
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: v_alignbit_b32 v5, v3, v17, 16
+; SI-NEXT: v_or_b32_e32 v31, v24, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v16, v16, v24
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_or_b32_e32 v38, v46, v9
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: ; implicit-def: $vgpr24
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
-; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v37, v9, v7
+; SI-NEXT: v_or_b32_e32 v37, v53, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v37
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v16, v16, v53
; SI-NEXT: ; implicit-def: $vgpr53
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_or_b32_e32 v17, v17, v55
+; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
+; SI-NEXT: ; implicit-def: $vgpr55
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v17, v54, v7
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; SI-NEXT: v_or_b32_e32 v38, v43, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr54
-; SI-NEXT: ; implicit-def: $vgpr43
-; SI-NEXT: v_or_b32_e32 v60, v16, v17
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v23, v56, v9
+; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
+; SI-NEXT: v_or_b32_e32 v19, v43, v7
+; SI-NEXT: v_or_b32_e32 v45, v17, v19
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v55
-; SI-NEXT: ; implicit-def: $vgpr55
+; SI-NEXT: v_or_b32_e32 v5, v5, v44
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
; SI-NEXT: v_or_b32_e32 v5, v5, v37
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v38
-; SI-NEXT: v_alignbit_b32 v7, v5, v17, 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v7, v5, v19, 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; SI-NEXT: v_or_b32_e32 v19, v45, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr45
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v16, v16, v47
-; SI-NEXT: ; implicit-def: $vgpr47
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
-; SI-NEXT: v_or_b32_e32 v41, v16, v19
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v37
+; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_or_b32_e32 v17, v17, v58
+; SI-NEXT: ; implicit-def: $vgpr58
+; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
+; SI-NEXT: v_or_b32_e32 v49, v17, v23
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
-; SI-NEXT: v_or_b32_e32 v7, v7, v57
-; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: v_or_b32_e32 v7, v7, v59
+; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_or_b32_e32 v7, v7, v38
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v48
-; SI-NEXT: v_alignbit_b32 v44, v7, v19, 16
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v9, v9, v61
-; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_or_b32_e32 v9, v9, v48
-; SI-NEXT: v_alignbit_b32 v31, v9, v20, 16
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v2
-; SI-NEXT: ; implicit-def: $vgpr2
-; SI-NEXT: ; implicit-def: $vgpr61
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v38
+; SI-NEXT: v_alignbit_b32 v9, v7, v23, 16
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_or_b32_e32 v14, v17, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v0, v16, v0
-; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_or_b32_e32 v0, v9, v0
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v39, v0, v20
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_or_b32_e32 v39, v57, v9
+; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: v_or_b32_e32 v9, v0, v39
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v28
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v39
+; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
+; SI-NEXT: v_or_b32_e32 v0, v60, v0
+; SI-NEXT: ; implicit-def: $vgpr60
+; SI-NEXT: v_alignbit_b32 v47, v9, v0, 16
+; SI-NEXT: v_or_b32_e32 v33, v14, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr20
+; SI-NEXT: ; implicit-def: $vgpr14
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v0, v0, v10
-; SI-NEXT: ; implicit-def: $vgpr10
+; SI-NEXT: v_or_b32_e32 v0, v0, v4
+; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: ; kill: killed $vgpr4
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v25, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v6
+; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: ; kill: killed $vgpr4
+; SI-NEXT: v_or_b32_e32 v23, v0, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v18
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr4
; SI-NEXT: v_or_b32_e32 v0, v0, v34
; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v19, v0, v4
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v56
-; SI-NEXT: ; implicit-def: $vgpr56
-; SI-NEXT: ; implicit-def: $vgpr4
-; SI-NEXT: v_or_b32_e32 v0, v0, v51
-; SI-NEXT: ; implicit-def: $vgpr51
+; SI-NEXT: v_or_b32_e32 v19, v0, v2
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v40
+; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: ; implicit-def: $vgpr2
+; SI-NEXT: v_or_b32_e32 v0, v0, v52
+; SI-NEXT: ; implicit-def: $vgpr52
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v17, v0, v8
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
+; SI-NEXT: v_or_b32_e32 v17, v0, v12
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
+; SI-NEXT: ; implicit-def: $vgpr12
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
@@ -68552,107 +68559,129 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB98_4
; SI-NEXT: ; %bb.3: ; %cmp.true
-; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
-; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_or_b32_e32 v1, v62, v1
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v59, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v56
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v40
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v16
-; SI-NEXT: v_or_b32_e32 v3, v51, v3
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v62
+; SI-NEXT: v_or_b32_e32 v3, v52, v3
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x300, v3
-; SI-NEXT: v_or_b32_e32 v5, v27, v5
+; SI-NEXT: v_or_b32_e32 v5, v29, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v8
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v61
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v33, v5
+; SI-NEXT: v_or_b32_e32 v5, v27, v5
+; SI-NEXT: s_movk_i32 s6, 0x300
; SI-NEXT: v_add_i32_e32 v17, vcc, 0x3000000, v3
; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v5
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v30
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v6
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v5, v21, v5
+; SI-NEXT: v_or_b32_e32 v5, v12, v5
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v6
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v18
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v42
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v41
; SI-NEXT: v_or_b32_e32 v3, v34, v3
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v4, v5
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v40
+; SI-NEXT: v_or_b32_e32 v2, v2, v5
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v42
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v26, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_or_b32_e32 v1, v24, v1
+; SI-NEXT: v_alignbit_b32 v20, v15, v17, 16
+; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v15
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v3, v18, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_alignbit_b32 v32, v15, v17, 16
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_or_b32_e32 v3, v30, v3
+; SI-NEXT: v_or_b32_e32 v2, v3, v2
+; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: v_alignbit_b32 v22, v13, v19, 16
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v13
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v4, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v1, v1, v2
+; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v1
+; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v25
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v8
+; SI-NEXT: v_or_b32_e32 v1, v16, v1
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v2, v63, v2
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v28
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v2, v60, v2
+; SI-NEXT: v_alignbit_b32 v32, v11, v23, 16
; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v15
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: v_or_b32_e32 v1, v14, v1
+; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_add_i32_e32 v33, vcc, 0x3000000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v20
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v58, v1
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v39, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v61, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v46, v1
+; SI-NEXT: v_or_b32_e32 v1, v57, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v9, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v31, v9, v39, 16
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v9
+; SI-NEXT: v_alignbit_b32 v47, v9, v33, 16
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v9
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v47, v0
+; SI-NEXT: v_or_b32_e32 v0, v58, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v45, v1
+; SI-NEXT: v_or_b32_e32 v1, v56, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v41, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v49, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
@@ -68660,75 +68689,52 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v57, v0
+; SI-NEXT: v_or_b32_e32 v0, v59, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v43, v1
+; SI-NEXT: v_or_b32_e32 v1, v46, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v7, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v44, v7, v41, 16
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v7
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v53, v0
+; SI-NEXT: v_or_b32_e32 v0, v55, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v54, v1
+; SI-NEXT: v_or_b32_e32 v1, v43, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v60, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v45, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v24, v4
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v28
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v63, v4
-; SI-NEXT: v_alignbit_b32 v26, v13, v19, 16
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v13
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v55, v0
+; SI-NEXT: v_or_b32_e32 v0, v44, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_or_b32_e32 v1, v53, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v5, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v3, v10, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_add_i32_e32 v25, vcc, 0x3000000, v3
-; SI-NEXT: v_alignbit_b32 v22, v11, v25, 16
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
@@ -68739,16 +68745,16 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v31, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v52, v0
+; SI-NEXT: v_or_b32_e32 v0, v54, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
@@ -68758,8 +68764,8 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v3
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -68774,10 +68780,10 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v10, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -68792,55 +68798,51 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v0
-; SI-NEXT: v_alignbit_b32 v0, v23, v12, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: v_add_i32_e32 v21, vcc, 0x3000000, v0
+; SI-NEXT: v_alignbit_b32 v0, v21, v10, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_alignbit_b32 v0, v3, v31, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_alignbit_b32 v0, v3, v14, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v5, v45, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_alignbit_b32 v0, v5, v60, 16
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v23
+; SI-NEXT: v_alignbit_b32 v0, v7, v49, 16
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v21
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
; SI-NEXT: .LBB98_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v12
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v10
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v35
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v47
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v44
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v31
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v32
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v22
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v20
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v23
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v21
; SI-NEXT: v_or_b32_e32 v1, v1, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v14
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v31
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v2, v2, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v36
; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v60
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_or_b32_e32 v4, v4, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v29
-; SI-NEXT: v_or_b32_e32 v5, v5, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v41
+; SI-NEXT: v_and_b32_e32 v4, 0xffff, v45
; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
@@ -68857,25 +68859,31 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
+; SI-NEXT: v_or_b32_e32 v4, v4, v6
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v37
+; SI-NEXT: v_or_b32_e32 v5, v5, v6
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v49
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v6, v6, v8
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v38
; SI-NEXT: v_or_b32_e32 v7, v7, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v39
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v33
; SI-NEXT: v_or_b32_e32 v8, v8, v10
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v39
; SI-NEXT: v_or_b32_e32 v9, v9, v10
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v25
+; SI-NEXT: v_and_b32_e32 v10, 0xffff, v23
; SI-NEXT: v_or_b32_e32 v10, v10, v12
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v48
; SI-NEXT: v_or_b32_e32 v11, v11, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v19
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v26
; SI-NEXT: v_or_b32_e32 v12, v12, v14
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v50
; SI-NEXT: v_or_b32_e32 v13, v13, v14
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v17
; SI-NEXT: v_or_b32_e32 v14, v14, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v50
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v51
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -68901,16 +68909,16 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:36
-; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
-; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:24
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:20
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:16
-; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:12
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:32
+; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
+; VI-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:20
+; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:16
+; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:12
+; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:4
; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
@@ -68937,18 +68945,18 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:100
; VI-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:96
; VI-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
-; VI-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:88
-; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:84
-; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:80
-; VI-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:72
-; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:68
-; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:64
-; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:60
-; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:56
-; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:88
+; VI-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:84
+; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:80
+; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:72
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:68
+; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:64
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:60
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:56
+; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:52
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:44
; VI-NEXT: v_mov_b32_e32 v17, v14
; VI-NEXT: s_waitcnt vmcnt(14)
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
@@ -68968,16 +68976,16 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_perm_b32 v8, v30, v55, s6
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; VI-NEXT: v_perm_b32 v9, v44, v42, s6
+; VI-NEXT: v_perm_b32 v9, v62, v59, s6
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; VI-NEXT: v_perm_b32 v10, v57, v46, s6
+; VI-NEXT: v_perm_b32 v10, v37, v35, s6
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: s_waitcnt vmcnt(2)
-; VI-NEXT: v_perm_b32 v11, v58, v47, s6
+; VI-NEXT: v_perm_b32 v11, v42, v40, s6
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; VI-NEXT: v_perm_b32 v12, v33, v62, s6
+; VI-NEXT: v_perm_b32 v12, v57, v46, s6
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
-; VI-NEXT: v_perm_b32 v13, v35, v34, s6
+; VI-NEXT: v_perm_b32 v13, v34, v33, s6
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v49, v48, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -68993,16 +69001,16 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr27
; VI-NEXT: ; implicit-def: $vgpr30
; VI-NEXT: ; implicit-def: $vgpr55
-; VI-NEXT: ; implicit-def: $vgpr44
+; VI-NEXT: ; implicit-def: $vgpr62
+; VI-NEXT: ; implicit-def: $vgpr59
+; VI-NEXT: ; implicit-def: $vgpr37
+; VI-NEXT: ; implicit-def: $vgpr35
; VI-NEXT: ; implicit-def: $vgpr42
+; VI-NEXT: ; implicit-def: $vgpr40
; VI-NEXT: ; implicit-def: $vgpr57
; VI-NEXT: ; implicit-def: $vgpr46
-; VI-NEXT: ; implicit-def: $vgpr58
-; VI-NEXT: ; implicit-def: $vgpr47
-; VI-NEXT: ; implicit-def: $vgpr33
-; VI-NEXT: ; implicit-def: $vgpr62
-; VI-NEXT: ; implicit-def: $vgpr35
; VI-NEXT: ; implicit-def: $vgpr34
+; VI-NEXT: ; implicit-def: $vgpr33
; VI-NEXT: ; implicit-def: $vgpr49
; VI-NEXT: ; implicit-def: $vgpr48
; VI-NEXT: ; implicit-def: $vgpr51
@@ -69055,17 +69063,17 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_perm_b32 v7, v28, v29, s6
; VI-NEXT: v_or_b32_e32 v7, v7, v8
-; VI-NEXT: v_perm_b32 v8, v41, v40, s6
+; VI-NEXT: v_perm_b32 v8, v45, v44, s6
; VI-NEXT: v_or_b32_e32 v8, v8, v9
-; VI-NEXT: v_perm_b32 v9, v45, v43, s6
+; VI-NEXT: v_perm_b32 v9, v63, v60, s6
; VI-NEXT: v_or_b32_e32 v9, v9, v10
-; VI-NEXT: v_perm_b32 v10, v61, v60, s6
+; VI-NEXT: v_perm_b32 v10, v61, v56, s6
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: v_perm_b32 v11, v59, v56, s6
+; VI-NEXT: v_perm_b32 v11, v43, v41, s6
; VI-NEXT: v_or_b32_e32 v11, v11, v12
-; VI-NEXT: v_perm_b32 v12, v31, v63, s6
+; VI-NEXT: v_perm_b32 v12, v58, v47, s6
; VI-NEXT: v_or_b32_e32 v12, v12, v13
-; VI-NEXT: v_perm_b32 v13, v37, v36, s6
+; VI-NEXT: v_perm_b32 v13, v36, v31, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
; VI-NEXT: v_perm_b32 v14, v39, v38, s6
; VI-NEXT: v_perm_b32 v16, v32, v54, s6
@@ -69110,18 +69118,18 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr25
; VI-NEXT: ; implicit-def: $vgpr28
; VI-NEXT: ; implicit-def: $vgpr29
-; VI-NEXT: ; implicit-def: $vgpr41
-; VI-NEXT: ; implicit-def: $vgpr40
; VI-NEXT: ; implicit-def: $vgpr45
-; VI-NEXT: ; implicit-def: $vgpr43
-; VI-NEXT: ; implicit-def: $vgpr61
+; VI-NEXT: ; implicit-def: $vgpr44
+; VI-NEXT: ; implicit-def: $vgpr63
; VI-NEXT: ; implicit-def: $vgpr60
-; VI-NEXT: ; implicit-def: $vgpr59
+; VI-NEXT: ; implicit-def: $vgpr61
; VI-NEXT: ; implicit-def: $vgpr56
-; VI-NEXT: ; implicit-def: $vgpr31
-; VI-NEXT: ; implicit-def: $vgpr63
-; VI-NEXT: ; implicit-def: $vgpr37
+; VI-NEXT: ; implicit-def: $vgpr43
+; VI-NEXT: ; implicit-def: $vgpr41
+; VI-NEXT: ; implicit-def: $vgpr58
+; VI-NEXT: ; implicit-def: $vgpr47
; VI-NEXT: ; implicit-def: $vgpr36
+; VI-NEXT: ; implicit-def: $vgpr31
; VI-NEXT: ; implicit-def: $vgpr39
; VI-NEXT: ; implicit-def: $vgpr38
; VI-NEXT: ; implicit-def: $vgpr53
@@ -69152,48 +69160,48 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_add_u16_sdwa v32, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_u16_sdwa v38, v1, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(10)
-; VI-NEXT: v_add_u16_e32 v0, 3, v37
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v36
-; VI-NEXT: v_or_b32_sdwa v36, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v36
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v31
+; VI-NEXT: v_or_b32_sdwa v31, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(8)
-; VI-NEXT: v_add_u16_e32 v0, 3, v35
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v34
+; VI-NEXT: v_add_u16_e32 v0, 3, v34
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v33
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v12, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(6)
-; VI-NEXT: v_add_u16_e32 v0, 3, v31
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v63
-; VI-NEXT: v_or_b32_sdwa v31, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v58
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v47
+; VI-NEXT: v_or_b32_sdwa v33, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(4)
-; VI-NEXT: v_add_u16_e32 v0, 3, v33
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v62
+; VI-NEXT: v_add_u16_e32 v0, 3, v57
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v46
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v11, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(2)
-; VI-NEXT: v_add_u16_e32 v0, 3, v59
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v56
-; VI-NEXT: v_or_b32_sdwa v33, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v43
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v41
+; VI-NEXT: v_or_b32_sdwa v34, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_add_u16_e32 v0, 3, v58
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v47
+; VI-NEXT: v_add_u16_e32 v0, 3, v42
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v40
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v10, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_u16_e32 v0, 3, v61
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v60
-; VI-NEXT: v_or_b32_sdwa v34, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: v_add_u16_e32 v0, 3, v57
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v46
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v56
+; VI-NEXT: v_or_b32_sdwa v36, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v37
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v35
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v9, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_u16_e32 v0, 3, v45
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v43
+; VI-NEXT: v_add_u16_e32 v0, 3, v63
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v60
; VI-NEXT: v_or_b32_sdwa v35, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: v_add_u16_e32 v0, 3, v44
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v42
+; VI-NEXT: v_add_u16_e32 v0, 3, v62
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v59
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v8, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_u16_e32 v0, 3, v41
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v40
+; VI-NEXT: v_add_u16_e32 v0, 3, v45
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v44
; VI-NEXT: v_or_b32_sdwa v37, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_e32 v0, 3, v30
; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v55
@@ -69246,13 +69254,13 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v8, v16, v8
; VI-NEXT: v_add_u16_e32 v16, 0x300, v35
; VI-NEXT: v_or_b32_e32 v9, v16, v9
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v34
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v36
; VI-NEXT: v_or_b32_e32 v10, v16, v10
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v33
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v34
; VI-NEXT: v_or_b32_e32 v11, v16, v11
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v31
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v33
; VI-NEXT: v_or_b32_e32 v12, v16, v12
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v36
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v31
; VI-NEXT: v_or_b32_e32 v13, v16, v13
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_lshlrev_b16_e32 v21, 8, v21
@@ -69360,16 +69368,16 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:36
-; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:24
-; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:20
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:36
+; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:20
+; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:4
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:120
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:116
-; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32
; GFX9-NEXT: v_mov_b32_e32 v19, v12
; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
@@ -69397,11 +69405,11 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:104
; GFX9-NEXT: buffer_load_ushort v11, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:128
-; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:108
-; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:100
-; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:92
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
+; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:108
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:88
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:60
@@ -69411,15 +69419,15 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; GFX9-NEXT: s_waitcnt vmcnt(22)
-; GFX9-NEXT: v_lshlrev_b16_e32 v56, 8, v1
+; GFX9-NEXT: v_lshlrev_b16_e32 v60, 8, v1
; GFX9-NEXT: s_waitcnt vmcnt(21)
-; GFX9-NEXT: v_lshlrev_b16_e32 v45, 8, v2
+; GFX9-NEXT: v_lshlrev_b16_e32 v58, 8, v2
; GFX9-NEXT: s_waitcnt vmcnt(20)
-; GFX9-NEXT: v_lshlrev_b16_e32 v61, 8, v3
+; GFX9-NEXT: v_lshlrev_b16_e32 v63, 8, v3
; GFX9-NEXT: s_waitcnt vmcnt(19)
; GFX9-NEXT: v_lshlrev_b16_e32 v38, 8, v4
; GFX9-NEXT: s_waitcnt vmcnt(18)
-; GFX9-NEXT: v_lshlrev_b16_e32 v33, 8, v5
+; GFX9-NEXT: v_lshlrev_b16_e32 v48, 8, v5
; GFX9-NEXT: s_waitcnt vmcnt(17)
; GFX9-NEXT: v_lshlrev_b16_e32 v37, 8, v6
; GFX9-NEXT: s_waitcnt vmcnt(16)
@@ -69445,33 +69453,33 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_mov_b32 s7, 0x5040100
; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
; GFX9-NEXT: v_perm_b32 v7, v26, v27, s6
-; GFX9-NEXT: v_perm_b32 v8, v30, v36, s6
-; GFX9-NEXT: v_or_b32_sdwa v9, v47, v45 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
-; GFX9-NEXT: v_perm_b32 v10, v63, v62, s6
+; GFX9-NEXT: v_perm_b32 v8, v30, v33, s6
+; GFX9-NEXT: v_or_b32_sdwa v9, v59, v58 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v10, v34, v36, s6
; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_or_b32_sdwa v11, v55, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v11, v55, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v12, v42, v41, s6
; GFX9-NEXT: v_or_b32_sdwa v13, v43, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
-; GFX9-NEXT: v_perm_b32 v14, v58, v57, s6
-; GFX9-NEXT: v_or_b32_sdwa v15, v34, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v47, v46, s6
+; GFX9-NEXT: v_or_b32_sdwa v15, v56, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: ; implicit-def: $vgpr23
; GFX9-NEXT: ; implicit-def: $vgpr26
; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr30
+; GFX9-NEXT: ; implicit-def: $vgpr33
+; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: ; implicit-def: $vgpr34
; GFX9-NEXT: ; implicit-def: $vgpr36
-; GFX9-NEXT: ; implicit-def: $vgpr47
-; GFX9-NEXT: ; implicit-def: $vgpr63
-; GFX9-NEXT: ; implicit-def: $vgpr62
; GFX9-NEXT: ; implicit-def: $vgpr55
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr41
; GFX9-NEXT: ; implicit-def: $vgpr43
+; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr46
+; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr58
-; GFX9-NEXT: ; implicit-def: $vgpr57
-; GFX9-NEXT: ; implicit-def: $vgpr34
-; GFX9-NEXT: ; implicit-def: $vgpr45
-; GFX9-NEXT: ; implicit-def: $vgpr33
+; GFX9-NEXT: ; implicit-def: $vgpr48
; GFX9-NEXT: ; implicit-def: $vgpr35
; GFX9-NEXT: ; implicit-def: $vgpr39
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -69523,19 +69531,19 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s7
; GFX9-NEXT: v_perm_b32 v7, v28, v29, s6
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s7
-; GFX9-NEXT: v_or_b32_sdwa v8, v59, v56 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v8, v61, v60 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v8, v9, v8, s7
-; GFX9-NEXT: v_or_b32_sdwa v9, v60, v61 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v9, v62, v63 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v9, v9, v10, s7
-; GFX9-NEXT: v_or_b32_sdwa v10, v48, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v10, v31, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v10, v11, v10, s7
; GFX9-NEXT: v_or_b32_sdwa v11, v40, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v11, v11, v12, s7
; GFX9-NEXT: v_or_b32_sdwa v12, v44, v51 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v12, v13, v12, s7
-; GFX9-NEXT: v_or_b32_sdwa v13, v46, v50 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v13, v45, v50 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v13, v13, v14, s7
-; GFX9-NEXT: v_or_b32_sdwa v14, v31, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v14, v57, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v14, v15, v14, s7
; GFX9-NEXT: v_or_b32_sdwa v15, v53, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v15, v15, v16, s7
@@ -69579,16 +69587,16 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
-; GFX9-NEXT: ; implicit-def: $vgpr59
-; GFX9-NEXT: ; implicit-def: $vgpr60
-; GFX9-NEXT: ; implicit-def: $vgpr48
+; GFX9-NEXT: ; implicit-def: $vgpr61
+; GFX9-NEXT: ; implicit-def: $vgpr62
+; GFX9-NEXT: ; implicit-def: $vgpr31
; GFX9-NEXT: ; implicit-def: $vgpr40
; GFX9-NEXT: ; implicit-def: $vgpr44
-; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr31
+; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr57
; GFX9-NEXT: ; implicit-def: $vgpr53
-; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr61
+; GFX9-NEXT: ; implicit-def: $vgpr60
+; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr38
; GFX9-NEXT: ; implicit-def: $vgpr37
; GFX9-NEXT: ; implicit-def: $vgpr51
@@ -69604,19 +69612,18 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v1, 3, v53
; GFX9-NEXT: v_or_b32_sdwa v1, v32, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: s_waitcnt vmcnt(9)
-; GFX9-NEXT: v_add_u16_e32 v2, 3, v31
; GFX9-NEXT: v_add_u16_e32 v15, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v31, 0x300, v1
+; GFX9-NEXT: v_add_u16_e32 v32, 0x300, v1
; GFX9-NEXT: s_waitcnt vmcnt(6)
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v58
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v57
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v47
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v46
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v46
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v2, 3, v57
; GFX9-NEXT: v_or_b32_sdwa v0, v50, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v3, 3, v34
-; GFX9-NEXT: v_add_u16_e32 v34, 0x300, v0
+; GFX9-NEXT: v_or_b32_sdwa v2, v49, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v49, 0x300, v0
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_add_u16_e32 v0, 3, v44
; GFX9-NEXT: v_or_b32_sdwa v0, v51, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -69632,33 +69639,32 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v0, 3, v40
; GFX9-NEXT: v_or_b32_sdwa v0, v37, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v37, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v48
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v31
; GFX9-NEXT: v_or_b32_sdwa v0, v38, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v0
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v0, 3, v55
-; GFX9-NEXT: v_or_b32_sdwa v0, v33, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v33, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v63
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v62
+; GFX9-NEXT: v_or_b32_sdwa v0, v48, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v31, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v34
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v36
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v60
-; GFX9-NEXT: v_or_b32_sdwa v0, v61, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v38, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v59
-; GFX9-NEXT: v_or_b32_sdwa v0, v56, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v62
+; GFX9-NEXT: v_or_b32_sdwa v0, v63, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v34, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v61
+; GFX9-NEXT: v_or_b32_sdwa v0, v60, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v47
-; GFX9-NEXT: v_or_b32_sdwa v0, v45, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_or_b32_sdwa v3, v39, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v39, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v59
+; GFX9-NEXT: v_or_b32_sdwa v0, v58, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v36, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v28
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v29
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v7, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v30
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v36
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v33
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v28, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v24
@@ -69683,21 +69689,22 @@ define <32 x i16> @bitcast_v64i8_to_v32i16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v4, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v18
-; GFX9-NEXT: v_add_u16_e32 v32, 0x300, v3
-; GFX9-NEXT: v_or_b32_sdwa v2, v49, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v3, 3, v56
+; GFX9-NEXT: v_or_b32_sdwa v3, v39, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v39, 0x300, v3
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v2
; GFX9-NEXT: s_mov_b32 s6, 0x5040100
; GFX9-NEXT: v_perm_b32 v5, v20, v5, s6
; GFX9-NEXT: v_perm_b32 v6, v24, v6, s6
; GFX9-NEXT: v_perm_b32 v7, v28, v7, s6
-; GFX9-NEXT: v_perm_b32 v8, v39, v8, s6
-; GFX9-NEXT: v_perm_b32 v9, v38, v9, s6
-; GFX9-NEXT: v_perm_b32 v10, v33, v10, s6
+; GFX9-NEXT: v_perm_b32 v8, v36, v8, s6
+; GFX9-NEXT: v_perm_b32 v9, v34, v9, s6
+; GFX9-NEXT: v_perm_b32 v10, v31, v10, s6
; GFX9-NEXT: v_perm_b32 v11, v37, v11, s6
; GFX9-NEXT: v_perm_b32 v12, v35, v12, s6
-; GFX9-NEXT: v_perm_b32 v13, v34, v13, s6
-; GFX9-NEXT: v_perm_b32 v14, v32, v14, s6
-; GFX9-NEXT: v_perm_b32 v15, v31, v15, s6
+; GFX9-NEXT: v_perm_b32 v13, v49, v13, s6
+; GFX9-NEXT: v_perm_b32 v14, v39, v14, s6
+; GFX9-NEXT: v_perm_b32 v15, v32, v15, s6
; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -71324,10 +71331,11 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s7, v21
; VI-NEXT: v_readfirstlane_b32 s9, v20
; VI-NEXT: v_readfirstlane_b32 s6, v19
-; VI-NEXT: v_readfirstlane_b32 s44, v18
-; VI-NEXT: v_readfirstlane_b32 s42, v17
-; VI-NEXT: v_readfirstlane_b32 s43, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; VI-NEXT: v_readfirstlane_b32 s56, v18
+; VI-NEXT: v_readfirstlane_b32 s43, v17
+; VI-NEXT: v_readfirstlane_b32 s46, v16
+; VI-NEXT: v_readfirstlane_b32 s45, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:4
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32
@@ -71347,24 +71355,23 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:12
-; VI-NEXT: v_readfirstlane_b32 s74, v15
-; VI-NEXT: v_readfirstlane_b32 s76, v14
-; VI-NEXT: v_readfirstlane_b32 s72, v13
-; VI-NEXT: v_readfirstlane_b32 s75, v12
-; VI-NEXT: v_readfirstlane_b32 s62, v11
-; VI-NEXT: v_readfirstlane_b32 s73, v10
-; VI-NEXT: v_readfirstlane_b32 s60, v9
-; VI-NEXT: v_readfirstlane_b32 s63, v8
-; VI-NEXT: v_readfirstlane_b32 s58, v7
-; VI-NEXT: v_readfirstlane_b32 s61, v6
-; VI-NEXT: v_readfirstlane_b32 s47, v5
-; VI-NEXT: v_readfirstlane_b32 s59, v4
-; VI-NEXT: v_readfirstlane_b32 s46, v3
-; VI-NEXT: v_readfirstlane_b32 s56, v2
-; VI-NEXT: v_readfirstlane_b32 s45, v1
-; VI-NEXT: v_readfirstlane_b32 s57, v0
+; VI-NEXT: v_readfirstlane_b32 s44, v15
+; VI-NEXT: v_readfirstlane_b32 s47, v14
+; VI-NEXT: v_readfirstlane_b32 s42, v13
+; VI-NEXT: v_readfirstlane_b32 s74, v11
+; VI-NEXT: v_readfirstlane_b32 s76, v10
+; VI-NEXT: v_readfirstlane_b32 s72, v9
+; VI-NEXT: v_readfirstlane_b32 s75, v8
+; VI-NEXT: v_readfirstlane_b32 s62, v7
+; VI-NEXT: v_readfirstlane_b32 s73, v6
+; VI-NEXT: v_readfirstlane_b32 s59, v5
+; VI-NEXT: v_readfirstlane_b32 s63, v4
+; VI-NEXT: v_readfirstlane_b32 s58, v3
+; VI-NEXT: v_readfirstlane_b32 s60, v2
+; VI-NEXT: v_readfirstlane_b32 s57, v1
+; VI-NEXT: v_readfirstlane_b32 s61, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB99_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -71386,41 +71393,41 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_perm_b32 v3, s26, v3, v11
; VI-NEXT: v_perm_b32 v2, s24, v2, v11
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; VI-NEXT: v_mov_b32_e32 v4, s45
+; VI-NEXT: v_mov_b32_e32 v4, s57
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s57, v4, v11
+; VI-NEXT: v_perm_b32 v4, s61, v4, v11
; VI-NEXT: v_perm_b32 v3, s28, v3, v11
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_mov_b32_e32 v5, s47
+; VI-NEXT: v_mov_b32_e32 v5, s59
; VI-NEXT: v_or_b32_e32 v3, v3, v4
-; VI-NEXT: v_mov_b32_e32 v4, s46
-; VI-NEXT: v_perm_b32 v5, s59, v5, v11
-; VI-NEXT: v_perm_b32 v4, s56, v4, v11
+; VI-NEXT: v_mov_b32_e32 v4, s58
+; VI-NEXT: v_perm_b32 v5, s63, v5, v11
+; VI-NEXT: v_perm_b32 v4, s60, v4, v11
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; VI-NEXT: v_mov_b32_e32 v6, s60
+; VI-NEXT: v_mov_b32_e32 v6, s72
; VI-NEXT: v_or_b32_e32 v4, v4, v5
-; VI-NEXT: v_mov_b32_e32 v5, s58
-; VI-NEXT: v_perm_b32 v6, s63, v6, v11
-; VI-NEXT: v_perm_b32 v5, s61, v5, v11
+; VI-NEXT: v_mov_b32_e32 v5, s62
+; VI-NEXT: v_perm_b32 v6, s75, v6, v11
+; VI-NEXT: v_perm_b32 v5, s73, v5, v11
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; VI-NEXT: v_mov_b32_e32 v7, s72
+; VI-NEXT: v_mov_b32_e32 v7, s42
; VI-NEXT: v_or_b32_e32 v5, v5, v6
-; VI-NEXT: v_mov_b32_e32 v6, s62
-; VI-NEXT: v_perm_b32 v7, s75, v7, v11
-; VI-NEXT: v_perm_b32 v6, s73, v6, v11
+; VI-NEXT: v_mov_b32_e32 v6, s74
+; VI-NEXT: v_perm_b32 v7, s45, v7, v11
+; VI-NEXT: v_perm_b32 v6, s76, v6, v11
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; VI-NEXT: v_mov_b32_e32 v8, s42
+; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
-; VI-NEXT: v_mov_b32_e32 v7, s74
-; VI-NEXT: v_perm_b32 v8, s43, v8, v11
-; VI-NEXT: v_perm_b32 v7, s76, v7, v11
+; VI-NEXT: v_mov_b32_e32 v7, s44
+; VI-NEXT: v_perm_b32 v8, s46, v8, v11
+; VI-NEXT: v_perm_b32 v7, s47, v7, v11
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s7
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s6
; VI-NEXT: v_perm_b32 v9, s9, v9, v11
-; VI-NEXT: v_perm_b32 v8, s44, v8, v11
+; VI-NEXT: v_perm_b32 v8, s56, v8, v11
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s10
; VI-NEXT: v_or_b32_e32 v8, v8, v9
@@ -71511,36 +71518,36 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: s_add_i32 s9, s9, 3
; VI-NEXT: v_mov_b32_e32 v1, s7
; VI-NEXT: v_perm_b32 v22, s9, v1, v0
-; VI-NEXT: s_add_i32 s44, s44, 3
+; VI-NEXT: s_add_i32 s56, s56, 3
; VI-NEXT: v_mov_b32_e32 v1, s6
-; VI-NEXT: v_perm_b32 v23, s44, v1, v0
-; VI-NEXT: s_add_i32 s43, s43, 3
+; VI-NEXT: v_perm_b32 v23, s56, v1, v0
+; VI-NEXT: s_add_i32 s46, s46, 3
+; VI-NEXT: v_mov_b32_e32 v1, s43
+; VI-NEXT: v_perm_b32 v7, s46, v1, v0
+; VI-NEXT: s_add_i32 s47, s47, 3
+; VI-NEXT: v_mov_b32_e32 v1, s44
+; VI-NEXT: v_perm_b32 v24, s47, v1, v0
+; VI-NEXT: s_add_i32 s45, s45, 3
; VI-NEXT: v_mov_b32_e32 v1, s42
-; VI-NEXT: v_perm_b32 v7, s43, v1, v0
+; VI-NEXT: v_perm_b32 v6, s45, v1, v0
; VI-NEXT: s_add_i32 s76, s76, 3
; VI-NEXT: v_mov_b32_e32 v1, s74
-; VI-NEXT: v_perm_b32 v24, s76, v1, v0
+; VI-NEXT: v_perm_b32 v25, s76, v1, v0
; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: v_mov_b32_e32 v1, s72
-; VI-NEXT: v_perm_b32 v6, s75, v1, v0
+; VI-NEXT: v_perm_b32 v5, s75, v1, v0
; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: v_mov_b32_e32 v1, s62
-; VI-NEXT: v_perm_b32 v25, s73, v1, v0
+; VI-NEXT: v_perm_b32 v26, s73, v1, v0
; VI-NEXT: s_add_i32 s63, s63, 3
-; VI-NEXT: v_mov_b32_e32 v1, s60
-; VI-NEXT: v_perm_b32 v5, s63, v1, v0
-; VI-NEXT: s_add_i32 s61, s61, 3
+; VI-NEXT: v_mov_b32_e32 v1, s59
+; VI-NEXT: v_perm_b32 v4, s63, v1, v0
+; VI-NEXT: s_add_i32 s60, s60, 3
; VI-NEXT: v_mov_b32_e32 v1, s58
-; VI-NEXT: v_perm_b32 v26, s61, v1, v0
-; VI-NEXT: s_add_i32 s59, s59, 3
-; VI-NEXT: v_mov_b32_e32 v1, s47
-; VI-NEXT: v_perm_b32 v4, s59, v1, v0
-; VI-NEXT: s_add_i32 s56, s56, 3
-; VI-NEXT: v_mov_b32_e32 v1, s46
-; VI-NEXT: v_perm_b32 v27, s56, v1, v0
-; VI-NEXT: s_add_i32 s57, s57, 3
-; VI-NEXT: v_mov_b32_e32 v1, s45
-; VI-NEXT: v_perm_b32 v3, s57, v1, v0
+; VI-NEXT: v_perm_b32 v27, s60, v1, v0
+; VI-NEXT: s_add_i32 s61, s61, 3
+; VI-NEXT: v_mov_b32_e32 v1, s57
+; VI-NEXT: v_perm_b32 v3, s61, v1, v0
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: v_mov_b32_e32 v1, s29
; VI-NEXT: v_perm_b32 v28, s28, v1, v0
@@ -71636,9 +71643,9 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9: ; %bb.0:
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX9-NEXT: v_readfirstlane_b32 s72, v30
-; GFX9-NEXT: v_readfirstlane_b32 s47, v29
+; GFX9-NEXT: v_readfirstlane_b32 s56, v29
; GFX9-NEXT: v_readfirstlane_b32 s57, v28
-; GFX9-NEXT: v_readfirstlane_b32 s60, v27
+; GFX9-NEXT: v_readfirstlane_b32 s61, v27
; GFX9-NEXT: v_readfirstlane_b32 s62, v26
; GFX9-NEXT: v_readfirstlane_b32 s7, v25
; GFX9-NEXT: v_readfirstlane_b32 s46, v24
@@ -71650,7 +71657,8 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s10, v18
; GFX9-NEXT: v_readfirstlane_b32 s11, v17
; GFX9-NEXT: v_readfirstlane_b32 s12, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; GFX9-NEXT: v_readfirstlane_b32 s15, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:4
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32
@@ -71671,23 +71679,22 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:12
; GFX9-NEXT: v_readfirstlane_b32 s13, v15
-; GFX9-NEXT: v_readfirstlane_b32 s76, v14
+; GFX9-NEXT: v_readfirstlane_b32 s73, v14
; GFX9-NEXT: v_readfirstlane_b32 s14, v13
-; GFX9-NEXT: v_readfirstlane_b32 s15, v12
; GFX9-NEXT: v_readfirstlane_b32 s40, v11
-; GFX9-NEXT: v_readfirstlane_b32 s75, v10
+; GFX9-NEXT: v_readfirstlane_b32 s76, v10
; GFX9-NEXT: v_readfirstlane_b32 s41, v9
; GFX9-NEXT: v_readfirstlane_b32 s42, v8
; GFX9-NEXT: v_readfirstlane_b32 s43, v7
-; GFX9-NEXT: v_readfirstlane_b32 s74, v6
+; GFX9-NEXT: v_readfirstlane_b32 s75, v6
; GFX9-NEXT: v_readfirstlane_b32 s44, v5
; GFX9-NEXT: v_readfirstlane_b32 s45, v4
-; GFX9-NEXT: v_readfirstlane_b32 s56, v3
-; GFX9-NEXT: v_readfirstlane_b32 s73, v2
-; GFX9-NEXT: v_readfirstlane_b32 s61, v1
+; GFX9-NEXT: v_readfirstlane_b32 s47, v3
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s60, v1
; GFX9-NEXT: v_readfirstlane_b32 s63, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB99_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -71709,7 +71716,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v11
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; GFX9-NEXT: v_mov_b32_e32 v4, s61
+; GFX9-NEXT: v_mov_b32_e32 v4, s60
; GFX9-NEXT: v_or_b32_sdwa v2, v3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s63, v4, v11
@@ -71717,27 +71724,27 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s44
; GFX9-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_mov_b32_e32 v4, s56
+; GFX9-NEXT: v_mov_b32_e32 v4, s47
; GFX9-NEXT: v_perm_b32 v5, s45, v5, v11
-; GFX9-NEXT: v_perm_b32 v4, s73, v4, v11
+; GFX9-NEXT: v_perm_b32 v4, s74, v4, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s41
; GFX9-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v5, s43
; GFX9-NEXT: v_perm_b32 v6, s42, v6, v11
-; GFX9-NEXT: v_perm_b32 v5, s74, v5, v11
+; GFX9-NEXT: v_perm_b32 v5, s75, v5, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s14
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v6, s40
; GFX9-NEXT: v_perm_b32 v7, s15, v7, v11
-; GFX9-NEXT: v_perm_b32 v6, s75, v6, v11
+; GFX9-NEXT: v_perm_b32 v6, s76, v6, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; GFX9-NEXT: v_mov_b32_e32 v8, s11
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v7, s13
; GFX9-NEXT: v_perm_b32 v8, s12, v8, v11
-; GFX9-NEXT: v_perm_b32 v7, s76, v7, v11
+; GFX9-NEXT: v_perm_b32 v7, s73, v7, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v10, 0x6050400
; GFX9-NEXT: v_mov_b32_e32 v9, s8
@@ -71755,8 +71762,8 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_or_b32_sdwa v9, v12, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v12, s62
; GFX9-NEXT: v_mov_b32_e32 v13, s57
-; GFX9-NEXT: v_perm_b32 v12, s60, v12, v10
-; GFX9-NEXT: v_perm_b32 v10, s47, v13, v10
+; GFX9-NEXT: v_perm_b32 v12, s61, v12, v10
+; GFX9-NEXT: v_perm_b32 v10, s56, v13, v10
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: s_mov_b32 s4, 0x6050400
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -71821,10 +71828,10 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_mov_b32_e32 v19, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v16, v0, v16, s4
; GFX9-NEXT: s_add_i32 s62, s62, 3
-; GFX9-NEXT: v_mov_b32_e32 v0, s60
+; GFX9-NEXT: v_mov_b32_e32 v0, s61
; GFX9-NEXT: v_perm_b32 v17, s62, v0, v19
; GFX9-NEXT: s_add_i32 s57, s57, 3
-; GFX9-NEXT: v_mov_b32_e32 v0, s47
+; GFX9-NEXT: v_mov_b32_e32 v0, s56
; GFX9-NEXT: v_perm_b32 v20, s57, v0, v19
; GFX9-NEXT: s_add_i32 s59, s59, 3
; GFX9-NEXT: v_mov_b32_e32 v0, s58
@@ -71860,44 +71867,44 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_or_b32_sdwa v2, v3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: s_add_i32 s63, s63, 3
-; GFX9-NEXT: v_mov_b32_e32 v4, s61
+; GFX9-NEXT: v_mov_b32_e32 v4, s60
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v19
; GFX9-NEXT: v_perm_b32 v4, s63, v4, v19
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s73, s73, 3
+; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_mov_b32_e32 v4, s56
+; GFX9-NEXT: v_mov_b32_e32 v4, s47
; GFX9-NEXT: s_add_i32 s45, s45, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s44
-; GFX9-NEXT: v_perm_b32 v4, s73, v4, v19
+; GFX9-NEXT: v_perm_b32 v4, s74, v4, v19
; GFX9-NEXT: v_perm_b32 v5, s45, v5, v19
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s74, s74, 3
+; GFX9-NEXT: s_add_i32 s75, s75, 3
; GFX9-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v5, s43
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s41
-; GFX9-NEXT: v_perm_b32 v5, s74, v5, v19
+; GFX9-NEXT: v_perm_b32 v5, s75, v5, v19
; GFX9-NEXT: v_perm_b32 v6, s42, v6, v19
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s75, s75, 3
+; GFX9-NEXT: s_add_i32 s76, s76, 3
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v6, s40
; GFX9-NEXT: s_add_i32 s15, s15, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s14
-; GFX9-NEXT: v_perm_b32 v6, s75, v6, v19
+; GFX9-NEXT: v_perm_b32 v6, s76, v6, v19
; GFX9-NEXT: v_perm_b32 v7, s15, v7, v19
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s76, s76, 3
+; GFX9-NEXT: s_add_i32 s73, s73, 3
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v7, s13
; GFX9-NEXT: s_add_i32 s12, s12, 3
; GFX9-NEXT: v_mov_b32_e32 v22, s11
-; GFX9-NEXT: v_perm_b32 v7, s76, v7, v19
+; GFX9-NEXT: v_perm_b32 v7, s73, v7, v19
; GFX9-NEXT: v_perm_b32 v22, s12, v22, v19
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -80753,370 +80760,365 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:24
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:16
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:48
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:88
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:96
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:16
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:80
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:120
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:128
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:112
-; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:8
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:40
-; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:92
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92
+; SI-NEXT: v_lshlrev_b32_e32 v1, 8, v1
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v1, 8, v9
+; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v27
+; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v58, 8, v25
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:108
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:100
; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v3
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v15
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v11
+; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v15
; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v23
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v11
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v7
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr3
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v52, 8, v13
-; SI-NEXT: v_lshlrev_b32_e32 v55, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v54, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v57, 8, v29
-; SI-NEXT: v_lshlrev_b32_e32 v45, 24, v27
-; SI-NEXT: v_lshlrev_b32_e32 v53, 8, v17
-; SI-NEXT: v_lshlrev_b32_e32 v47, 8, v25
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v54, 8, v13
+; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v21
+; SI-NEXT: v_lshlrev_b32_e32 v53, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v59, 8, v29
+; SI-NEXT: v_lshlrev_b32_e32 v55, 8, v17
; SI-NEXT: ; kill: killed $vgpr3
; SI-NEXT: ; implicit-def: $vgpr3
; SI-NEXT: ; implicit-def: $vgpr5
-; SI-NEXT: ; implicit-def: $vgpr23
+; SI-NEXT: ; implicit-def: $vgpr7
+; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; kill: killed $vgpr3
; SI-NEXT: ; implicit-def: $vgpr3
-; SI-NEXT: ; implicit-def: $vgpr60
+; SI-NEXT: ; implicit-def: $vgpr45
; SI-NEXT: ; kill: killed $vgpr5
; SI-NEXT: ; implicit-def: $vgpr5
-; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr41
-; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; kill: killed $vgpr7
; SI-NEXT: ; implicit-def: $vgpr7
-; SI-NEXT: ; implicit-def: $vgpr39
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: ; implicit-def: $vgpr9
+; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr11
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: ; implicit-def: $vgpr13
-; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: ; implicit-def: $vgpr15
-; SI-NEXT: ; implicit-def: $vgpr50
-; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:124
+; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; SI-NEXT: v_lshlrev_b32_e32 v43, 24, v2
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v1
-; SI-NEXT: v_lshlrev_b32_e32 v46, 24, v6
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v9
-; SI-NEXT: v_lshlrev_b32_e32 v61, 8, v4
-; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v18
-; SI-NEXT: v_lshlrev_b32_e32 v18, 8, v20
-; SI-NEXT: v_lshlrev_b32_e32 v58, 24, v10
-; SI-NEXT: v_lshlrev_b32_e32 v62, 8, v12
-; SI-NEXT: v_lshlrev_b32_e32 v33, 8, v28
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:12
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v14
-; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v22
-; SI-NEXT: v_lshlrev_b32_e32 v4, 24, v26
+; SI-NEXT: v_lshlrev_b32_e32 v46, 24, v2
+; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v4
+; SI-NEXT: v_lshlrev_b32_e32 v57, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v60, 24, v8
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:124
+; SI-NEXT: v_lshlrev_b32_e32 v63, 24, v12
+; SI-NEXT: v_lshlrev_b32_e32 v24, 24, v14
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v18
+; SI-NEXT: v_lshlrev_b32_e32 v30, 24, v20
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v22
+; SI-NEXT: v_lshlrev_b32_e32 v27, 8, v28
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:60
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:12
+; SI-NEXT: v_lshlrev_b32_e32 v12, 24, v31
+; SI-NEXT: v_lshlrev_b32_e32 v29, 24, v32
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v33
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v31
-; SI-NEXT: v_lshlrev_b32_e32 v27, 24, v32
-; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:76
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v0, 8, v34
-; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v35
-; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v36
-; SI-NEXT: v_lshlrev_b32_e32 v51, 8, v37
-; SI-NEXT: ; implicit-def: $vgpr12
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v34
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v35
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:84
+; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:76
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:36
+; SI-NEXT: v_lshlrev_b32_e32 v52, 8, v36
; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr14
-; SI-NEXT: ; implicit-def: $vgpr36
-; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr9
-; SI-NEXT: ; implicit-def: $vgpr22
-; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr32
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
+; SI-NEXT: ; implicit-def: $vgpr22
+; SI-NEXT: ; implicit-def: $vgpr20
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:268 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:28
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:20
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v56, off, s[0:3], s32 offset:100
+; SI-NEXT: v_lshlrev_b32_e32 v16, 8, v10
+; SI-NEXT: ; implicit-def: $vgpr10
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB106_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v13, 0xff, v40
-; SI-NEXT: v_or_b32_e32 v13, v13, v18
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v8, v8, v33
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_or_b32_e32 v1, v1, v62
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v59, v2
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_or_b32_e32 v1, v24, v1
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v25
+; SI-NEXT: v_or_b32_e32 v11, v11, v16
+; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
+; SI-NEXT: v_or_b32_e32 v6, v12, v6
+; SI-NEXT: v_and_b32_e32 v12, 0xff, v62
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_or_b32_e32 v12, v29, v12
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v42
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_or_b32_e32 v13, v13, v26
+; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_or_b32_e32 v8, v63, v8
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: v_or_b32_e32 v11, v11, v8
+; SI-NEXT: v_alignbit_b32 v32, v11, v1, 16
+; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v8
+; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v6
+; SI-NEXT: ; implicit-def: $vgpr8
+; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: ; implicit-def: $vgpr59
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr63
+; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_and_b32_e32 v24, 0xff, v24
; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_or_b32_e32 v24, v24, v25
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_or_b32_e32 v18, v24, v15
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v42
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_or_b32_e32 v4, v4, v15
-; SI-NEXT: v_and_b32_e32 v15, 0xff, v30
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_or_b32_e32 v21, v21, v15
-; SI-NEXT: v_or_b32_e32 v15, v8, v21
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_or_b32_e32 v8, v27, v8
-; SI-NEXT: v_or_b32_e32 v13, v13, v18
-; SI-NEXT: v_alignbit_b32 v26, v13, v4, 16
-; SI-NEXT: v_alignbit_b32 v32, v15, v8, 16
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v18
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v21
-; SI-NEXT: ; implicit-def: $vgpr42
+; SI-NEXT: v_or_b32_e32 v16, v30, v15
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v41
+; SI-NEXT: ; implicit-def: $vgpr41
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr18
-; SI-NEXT: ; implicit-def: $vgpr21
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_or_b32_e32 v13, v13, v16
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v16
+; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: v_or_b32_e32 v2, v2, v15
+; SI-NEXT: v_and_b32_e32 v15, 0xff, v61
+; SI-NEXT: ; implicit-def: $vgpr61
+; SI-NEXT: v_or_b32_e32 v15, v15, v27
+; SI-NEXT: v_alignbit_b32 v22, v13, v2, 16
; SI-NEXT: ; implicit-def: $vgpr27
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v48, v46, v11
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v20
-; SI-NEXT: ; implicit-def: $vgpr46
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v20, v58, v11
-; SI-NEXT: v_or_b32_e32 v11, v1, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v28
-; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: ; implicit-def: $vgpr58
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v1, v63, v1
-; SI-NEXT: ; implicit-def: $vgpr63
-; SI-NEXT: v_alignbit_b32 v22, v11, v1, 16
+; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
+; SI-NEXT: v_or_b32_e32 v15, v15, v6
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: v_alignbit_b32 v20, v15, v12, 16
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; SI-NEXT: v_or_b32_e32 v29, v7, v5
+; SI-NEXT: v_or_b32_e32 v35, v7, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v23, v3, v29
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v21, v3, v35
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v29
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_or_b32_e32 v16, v16, v24
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v35
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_or_b32_e32 v12, v7, v5
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_or_b32_e32 v10, v7, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v52
+; SI-NEXT: v_or_b32_e32 v5, v5, v54
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v36, v7, v3
-; SI-NEXT: v_alignbit_b32 v3, v23, v12, 16
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v3, v21, v10, 16
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
-; SI-NEXT: v_or_b32_e32 v12, v16, v12
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v10, v24, v10
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; SI-NEXT: v_or_b32_e32 v3, v3, v36
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v36
-; SI-NEXT: ; implicit-def: $vgpr52
+; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_or_b32_e32 v14, v7, v5
+; SI-NEXT: v_or_b32_e32 v17, v7, v5
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v5, v3, v14, 16
-; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v24, 0xff, v24
+; SI-NEXT: v_or_b32_e32 v24, v24, v25
+; SI-NEXT: v_and_b32_e32 v24, 0xffff, v24
+; SI-NEXT: v_alignbit_b32 v5, v3, v17, 16
+; SI-NEXT: v_or_b32_e32 v31, v24, v17
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
+; SI-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v16, v16, v24
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_or_b32_e32 v38, v46, v9
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: ; implicit-def: $vgpr24
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
-; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v37, v9, v7
+; SI-NEXT: v_or_b32_e32 v37, v53, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v37
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v16, v16, v53
; SI-NEXT: ; implicit-def: $vgpr53
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_or_b32_e32 v17, v17, v55
+; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
+; SI-NEXT: ; implicit-def: $vgpr55
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v17, v54, v7
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; SI-NEXT: v_or_b32_e32 v38, v43, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr54
-; SI-NEXT: ; implicit-def: $vgpr43
-; SI-NEXT: v_or_b32_e32 v60, v16, v17
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v23, v56, v9
+; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
+; SI-NEXT: v_or_b32_e32 v19, v43, v7
+; SI-NEXT: v_or_b32_e32 v45, v17, v19
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v5, v55
-; SI-NEXT: ; implicit-def: $vgpr55
+; SI-NEXT: v_or_b32_e32 v5, v5, v44
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
; SI-NEXT: v_or_b32_e32 v5, v5, v37
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v38
-; SI-NEXT: v_alignbit_b32 v7, v5, v17, 16
-; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v7, v5, v19, 16
+; SI-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; SI-NEXT: v_or_b32_e32 v19, v45, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr45
-; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v16, v16, v47
-; SI-NEXT: ; implicit-def: $vgpr47
-; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
-; SI-NEXT: v_or_b32_e32 v41, v16, v19
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v37
+; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_or_b32_e32 v17, v17, v58
+; SI-NEXT: ; implicit-def: $vgpr58
+; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
+; SI-NEXT: v_or_b32_e32 v49, v17, v23
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
-; SI-NEXT: v_or_b32_e32 v7, v7, v57
-; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: v_or_b32_e32 v7, v7, v59
+; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_or_b32_e32 v7, v7, v38
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v48
-; SI-NEXT: v_alignbit_b32 v44, v7, v19, 16
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v9, v9, v61
-; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_or_b32_e32 v9, v9, v48
-; SI-NEXT: v_alignbit_b32 v31, v9, v20, 16
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v2
-; SI-NEXT: ; implicit-def: $vgpr2
-; SI-NEXT: ; implicit-def: $vgpr61
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v38
+; SI-NEXT: v_alignbit_b32 v9, v7, v23, 16
+; SI-NEXT: buffer_store_dword v9, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_and_b32_e32 v17, 0xff, v17
+; SI-NEXT: v_or_b32_e32 v14, v17, v14
+; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v16, 0xff, v16
-; SI-NEXT: v_or_b32_e32 v0, v16, v0
-; SI-NEXT: ; implicit-def: $vgpr16
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_or_b32_e32 v0, v9, v0
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v39, v0, v20
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_or_b32_e32 v39, v57, v9
+; SI-NEXT: ; implicit-def: $vgpr57
+; SI-NEXT: v_or_b32_e32 v9, v0, v39
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v28
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v39
+; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
+; SI-NEXT: v_or_b32_e32 v0, v60, v0
+; SI-NEXT: ; implicit-def: $vgpr60
+; SI-NEXT: v_alignbit_b32 v47, v9, v0, 16
+; SI-NEXT: v_or_b32_e32 v33, v14, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr20
+; SI-NEXT: ; implicit-def: $vgpr14
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: v_or_b32_e32 v0, v0, v10
-; SI-NEXT: ; implicit-def: $vgpr10
+; SI-NEXT: v_or_b32_e32 v0, v0, v4
+; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: ; kill: killed $vgpr4
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v25, v0, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v6
+; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: ; kill: killed $vgpr4
+; SI-NEXT: v_or_b32_e32 v23, v0, v1
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v18
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr4
; SI-NEXT: v_or_b32_e32 v0, v0, v34
; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v19, v0, v4
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v56
-; SI-NEXT: ; implicit-def: $vgpr56
-; SI-NEXT: ; implicit-def: $vgpr4
-; SI-NEXT: v_or_b32_e32 v0, v0, v51
-; SI-NEXT: ; implicit-def: $vgpr51
+; SI-NEXT: v_or_b32_e32 v19, v0, v2
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v40
+; SI-NEXT: ; implicit-def: $vgpr40
+; SI-NEXT: ; implicit-def: $vgpr2
+; SI-NEXT: v_or_b32_e32 v0, v0, v52
+; SI-NEXT: ; implicit-def: $vgpr52
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v17, v0, v8
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
+; SI-NEXT: v_or_b32_e32 v17, v0, v12
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
+; SI-NEXT: ; implicit-def: $vgpr12
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
@@ -81172,107 +81174,129 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB106_4
; SI-NEXT: ; %bb.3: ; %cmp.true
-; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
-; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_or_b32_e32 v1, v62, v1
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v59, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v56
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v40
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v16
-; SI-NEXT: v_or_b32_e32 v3, v51, v3
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v62
+; SI-NEXT: v_or_b32_e32 v3, v52, v3
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x300, v3
-; SI-NEXT: v_or_b32_e32 v5, v27, v5
+; SI-NEXT: v_or_b32_e32 v5, v29, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v8
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v61
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v5, v33, v5
+; SI-NEXT: v_or_b32_e32 v5, v27, v5
+; SI-NEXT: s_movk_i32 s6, 0x300
; SI-NEXT: v_add_i32_e32 v17, vcc, 0x3000000, v3
; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v5
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v30
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v6
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v5, v21, v5
+; SI-NEXT: v_or_b32_e32 v5, v12, v5
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v6
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v18
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v42
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v41
; SI-NEXT: v_or_b32_e32 v3, v34, v3
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v4, v5
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v40
+; SI-NEXT: v_or_b32_e32 v2, v2, v5
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v19, vcc, 0x3000000, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v42
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v26, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
+; SI-NEXT: v_or_b32_e32 v1, v24, v1
+; SI-NEXT: v_alignbit_b32 v20, v15, v17, 16
+; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v15
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v3, v18, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_alignbit_b32 v32, v15, v17, 16
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_or_b32_e32 v3, v30, v3
+; SI-NEXT: v_or_b32_e32 v2, v3, v2
+; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: v_alignbit_b32 v22, v13, v19, 16
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v13
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_or_b32_e32 v2, v4, v2
+; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
+; SI-NEXT: v_or_b32_e32 v1, v1, v2
+; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v1
+; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v25
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v8
+; SI-NEXT: v_or_b32_e32 v1, v16, v1
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v2, v63, v2
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v28
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v2, v60, v2
+; SI-NEXT: v_alignbit_b32 v32, v11, v23, 16
; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v15
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: v_or_b32_e32 v1, v14, v1
+; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
+; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_add_i32_e32 v33, vcc, 0x3000000, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v20
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v58, v1
-; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v39, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v61, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v46, v1
+; SI-NEXT: v_or_b32_e32 v1, v57, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v9, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v31, v9, v39, 16
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v9
+; SI-NEXT: v_alignbit_b32 v47, v9, v33, 16
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v9
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v47, v0
+; SI-NEXT: v_or_b32_e32 v0, v58, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v45, v1
+; SI-NEXT: v_or_b32_e32 v1, v56, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v41, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v49, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
@@ -81280,75 +81304,52 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v57, v0
+; SI-NEXT: v_or_b32_e32 v0, v59, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v43, v1
+; SI-NEXT: v_or_b32_e32 v1, v46, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v7, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: v_alignbit_b32 v44, v7, v41, 16
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v7
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v53, v0
+; SI-NEXT: v_or_b32_e32 v0, v55, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v54, v1
+; SI-NEXT: v_or_b32_e32 v1, v43, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v60, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v45, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v24, v4
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v28
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v63, v4
-; SI-NEXT: v_alignbit_b32 v26, v13, v19, 16
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v13
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v55, v0
+; SI-NEXT: v_or_b32_e32 v0, v44, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
+; SI-NEXT: v_or_b32_e32 v1, v53, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_add_i32_e32 v5, vcc, 0x3000000, v0
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_or_b32_e32 v3, v10, v3
-; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
-; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
-; SI-NEXT: v_add_i32_e32 v25, vcc, 0x3000000, v3
-; SI-NEXT: v_alignbit_b32 v22, v11, v25, 16
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v5
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
@@ -81359,16 +81360,16 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v31, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v1
-; SI-NEXT: v_or_b32_e32 v0, v52, v0
+; SI-NEXT: v_or_b32_e32 v0, v54, v0
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
@@ -81378,8 +81379,8 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v3
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -81394,10 +81395,10 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v0
+; SI-NEXT: v_add_i32_e32 v10, vcc, 0x3000000, v0
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
@@ -81412,55 +81413,51 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_add_i32_e32 v23, vcc, 0x3000000, v0
-; SI-NEXT: v_alignbit_b32 v0, v23, v12, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
+; SI-NEXT: v_add_i32_e32 v21, vcc, 0x3000000, v0
+; SI-NEXT: v_alignbit_b32 v0, v21, v10, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_alignbit_b32 v0, v3, v31, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_alignbit_b32 v0, v3, v14, 16
-; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: v_alignbit_b32 v0, v5, v45, 16
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_alignbit_b32 v0, v5, v60, 16
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v23
+; SI-NEXT: v_alignbit_b32 v0, v7, v49, 16
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v21
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
; SI-NEXT: .LBB106_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v0, 0xffff, v12
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v10
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v35
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v47
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v44
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v31
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v32
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v22
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v20
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v23
+; SI-NEXT: v_and_b32_e32 v1, 0xffff, v21
; SI-NEXT: v_or_b32_e32 v1, v1, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v14
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_and_b32_e32 v2, 0xffff, v31
+; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v2, v2, v4
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v36
; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v60
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_or_b32_e32 v4, v4, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v29
-; SI-NEXT: v_or_b32_e32 v5, v5, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v41
+; SI-NEXT: v_and_b32_e32 v4, 0xffff, v45
; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
@@ -81477,25 +81474,31 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
+; SI-NEXT: v_or_b32_e32 v4, v4, v6
+; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v37
+; SI-NEXT: v_or_b32_e32 v5, v5, v6
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v49
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v6, v6, v8
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v38
; SI-NEXT: v_or_b32_e32 v7, v7, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v39
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v33
; SI-NEXT: v_or_b32_e32 v8, v8, v10
-; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v39
; SI-NEXT: v_or_b32_e32 v9, v9, v10
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v25
+; SI-NEXT: v_and_b32_e32 v10, 0xffff, v23
; SI-NEXT: v_or_b32_e32 v10, v10, v12
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v48
; SI-NEXT: v_or_b32_e32 v11, v11, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v19
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v26
; SI-NEXT: v_or_b32_e32 v12, v12, v14
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v50
; SI-NEXT: v_or_b32_e32 v13, v13, v14
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v17
; SI-NEXT: v_or_b32_e32 v14, v14, v16
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v50
+; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v51
; SI-NEXT: v_or_b32_e32 v15, v15, v16
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -81521,16 +81524,16 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:36
-; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
-; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:24
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:20
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:16
-; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:12
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:32
+; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
+; VI-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:20
+; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:16
+; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:12
+; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:4
; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
@@ -81557,18 +81560,18 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:100
; VI-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:96
; VI-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
-; VI-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:88
-; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:84
-; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:80
-; VI-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:72
-; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:68
-; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:64
-; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:60
-; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:56
-; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:88
+; VI-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:84
+; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:80
+; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:72
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:68
+; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:64
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:60
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:56
+; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:52
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:44
; VI-NEXT: v_mov_b32_e32 v17, v14
; VI-NEXT: s_waitcnt vmcnt(14)
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
@@ -81588,16 +81591,16 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_perm_b32 v8, v30, v55, s6
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; VI-NEXT: v_perm_b32 v9, v44, v42, s6
+; VI-NEXT: v_perm_b32 v9, v62, v59, s6
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; VI-NEXT: v_perm_b32 v10, v57, v46, s6
+; VI-NEXT: v_perm_b32 v10, v37, v35, s6
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: s_waitcnt vmcnt(2)
-; VI-NEXT: v_perm_b32 v11, v58, v47, s6
+; VI-NEXT: v_perm_b32 v11, v42, v40, s6
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; VI-NEXT: v_perm_b32 v12, v33, v62, s6
+; VI-NEXT: v_perm_b32 v12, v57, v46, s6
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
-; VI-NEXT: v_perm_b32 v13, v35, v34, s6
+; VI-NEXT: v_perm_b32 v13, v34, v33, s6
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v49, v48, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -81613,16 +81616,16 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr27
; VI-NEXT: ; implicit-def: $vgpr30
; VI-NEXT: ; implicit-def: $vgpr55
-; VI-NEXT: ; implicit-def: $vgpr44
+; VI-NEXT: ; implicit-def: $vgpr62
+; VI-NEXT: ; implicit-def: $vgpr59
+; VI-NEXT: ; implicit-def: $vgpr37
+; VI-NEXT: ; implicit-def: $vgpr35
; VI-NEXT: ; implicit-def: $vgpr42
+; VI-NEXT: ; implicit-def: $vgpr40
; VI-NEXT: ; implicit-def: $vgpr57
; VI-NEXT: ; implicit-def: $vgpr46
-; VI-NEXT: ; implicit-def: $vgpr58
-; VI-NEXT: ; implicit-def: $vgpr47
-; VI-NEXT: ; implicit-def: $vgpr33
-; VI-NEXT: ; implicit-def: $vgpr62
-; VI-NEXT: ; implicit-def: $vgpr35
; VI-NEXT: ; implicit-def: $vgpr34
+; VI-NEXT: ; implicit-def: $vgpr33
; VI-NEXT: ; implicit-def: $vgpr49
; VI-NEXT: ; implicit-def: $vgpr48
; VI-NEXT: ; implicit-def: $vgpr51
@@ -81675,17 +81678,17 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_perm_b32 v7, v28, v29, s6
; VI-NEXT: v_or_b32_e32 v7, v7, v8
-; VI-NEXT: v_perm_b32 v8, v41, v40, s6
+; VI-NEXT: v_perm_b32 v8, v45, v44, s6
; VI-NEXT: v_or_b32_e32 v8, v8, v9
-; VI-NEXT: v_perm_b32 v9, v45, v43, s6
+; VI-NEXT: v_perm_b32 v9, v63, v60, s6
; VI-NEXT: v_or_b32_e32 v9, v9, v10
-; VI-NEXT: v_perm_b32 v10, v61, v60, s6
+; VI-NEXT: v_perm_b32 v10, v61, v56, s6
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: v_perm_b32 v11, v59, v56, s6
+; VI-NEXT: v_perm_b32 v11, v43, v41, s6
; VI-NEXT: v_or_b32_e32 v11, v11, v12
-; VI-NEXT: v_perm_b32 v12, v31, v63, s6
+; VI-NEXT: v_perm_b32 v12, v58, v47, s6
; VI-NEXT: v_or_b32_e32 v12, v12, v13
-; VI-NEXT: v_perm_b32 v13, v37, v36, s6
+; VI-NEXT: v_perm_b32 v13, v36, v31, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
; VI-NEXT: v_perm_b32 v14, v39, v38, s6
; VI-NEXT: v_perm_b32 v16, v32, v54, s6
@@ -81730,18 +81733,18 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr25
; VI-NEXT: ; implicit-def: $vgpr28
; VI-NEXT: ; implicit-def: $vgpr29
-; VI-NEXT: ; implicit-def: $vgpr41
-; VI-NEXT: ; implicit-def: $vgpr40
; VI-NEXT: ; implicit-def: $vgpr45
-; VI-NEXT: ; implicit-def: $vgpr43
-; VI-NEXT: ; implicit-def: $vgpr61
+; VI-NEXT: ; implicit-def: $vgpr44
+; VI-NEXT: ; implicit-def: $vgpr63
; VI-NEXT: ; implicit-def: $vgpr60
-; VI-NEXT: ; implicit-def: $vgpr59
+; VI-NEXT: ; implicit-def: $vgpr61
; VI-NEXT: ; implicit-def: $vgpr56
-; VI-NEXT: ; implicit-def: $vgpr31
-; VI-NEXT: ; implicit-def: $vgpr63
-; VI-NEXT: ; implicit-def: $vgpr37
+; VI-NEXT: ; implicit-def: $vgpr43
+; VI-NEXT: ; implicit-def: $vgpr41
+; VI-NEXT: ; implicit-def: $vgpr58
+; VI-NEXT: ; implicit-def: $vgpr47
; VI-NEXT: ; implicit-def: $vgpr36
+; VI-NEXT: ; implicit-def: $vgpr31
; VI-NEXT: ; implicit-def: $vgpr39
; VI-NEXT: ; implicit-def: $vgpr38
; VI-NEXT: ; implicit-def: $vgpr53
@@ -81772,48 +81775,48 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_add_u16_sdwa v32, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_u16_sdwa v38, v1, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(10)
-; VI-NEXT: v_add_u16_e32 v0, 3, v37
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v36
-; VI-NEXT: v_or_b32_sdwa v36, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v36
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v31
+; VI-NEXT: v_or_b32_sdwa v31, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(8)
-; VI-NEXT: v_add_u16_e32 v0, 3, v35
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v34
+; VI-NEXT: v_add_u16_e32 v0, 3, v34
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v33
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v12, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(6)
-; VI-NEXT: v_add_u16_e32 v0, 3, v31
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v63
-; VI-NEXT: v_or_b32_sdwa v31, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v58
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v47
+; VI-NEXT: v_or_b32_sdwa v33, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(4)
-; VI-NEXT: v_add_u16_e32 v0, 3, v33
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v62
+; VI-NEXT: v_add_u16_e32 v0, 3, v57
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v46
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v11, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(2)
-; VI-NEXT: v_add_u16_e32 v0, 3, v59
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v56
-; VI-NEXT: v_or_b32_sdwa v33, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v43
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v41
+; VI-NEXT: v_or_b32_sdwa v34, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_add_u16_e32 v0, 3, v58
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v47
+; VI-NEXT: v_add_u16_e32 v0, 3, v42
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v40
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v10, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_u16_e32 v0, 3, v61
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v60
-; VI-NEXT: v_or_b32_sdwa v34, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: v_add_u16_e32 v0, 3, v57
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v46
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v56
+; VI-NEXT: v_or_b32_sdwa v36, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v37
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v35
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v9, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_u16_e32 v0, 3, v45
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v43
+; VI-NEXT: v_add_u16_e32 v0, 3, v63
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v60
; VI-NEXT: v_or_b32_sdwa v35, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: v_add_u16_e32 v0, 3, v44
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v42
+; VI-NEXT: v_add_u16_e32 v0, 3, v62
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v59
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v8, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_u16_e32 v0, 3, v41
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v40
+; VI-NEXT: v_add_u16_e32 v0, 3, v45
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v44
; VI-NEXT: v_or_b32_sdwa v37, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_e32 v0, 3, v30
; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v55
@@ -81866,13 +81869,13 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v8, v16, v8
; VI-NEXT: v_add_u16_e32 v16, 0x300, v35
; VI-NEXT: v_or_b32_e32 v9, v16, v9
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v34
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v36
; VI-NEXT: v_or_b32_e32 v10, v16, v10
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v33
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v34
; VI-NEXT: v_or_b32_e32 v11, v16, v11
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v31
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v33
; VI-NEXT: v_or_b32_e32 v12, v16, v12
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v36
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v31
; VI-NEXT: v_or_b32_e32 v13, v16, v13
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_lshlrev_b16_e32 v21, 8, v21
@@ -81980,16 +81983,16 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:36
-; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:24
-; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:20
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:36
+; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:20
+; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:4
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:120
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:116
-; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32
; GFX9-NEXT: v_mov_b32_e32 v19, v12
; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
@@ -82017,11 +82020,11 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:104
; GFX9-NEXT: buffer_load_ushort v11, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:128
-; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:108
-; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:100
-; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:92
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
+; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:108
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:88
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:60
@@ -82031,15 +82034,15 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; GFX9-NEXT: s_waitcnt vmcnt(22)
-; GFX9-NEXT: v_lshlrev_b16_e32 v56, 8, v1
+; GFX9-NEXT: v_lshlrev_b16_e32 v60, 8, v1
; GFX9-NEXT: s_waitcnt vmcnt(21)
-; GFX9-NEXT: v_lshlrev_b16_e32 v45, 8, v2
+; GFX9-NEXT: v_lshlrev_b16_e32 v58, 8, v2
; GFX9-NEXT: s_waitcnt vmcnt(20)
-; GFX9-NEXT: v_lshlrev_b16_e32 v61, 8, v3
+; GFX9-NEXT: v_lshlrev_b16_e32 v63, 8, v3
; GFX9-NEXT: s_waitcnt vmcnt(19)
; GFX9-NEXT: v_lshlrev_b16_e32 v38, 8, v4
; GFX9-NEXT: s_waitcnt vmcnt(18)
-; GFX9-NEXT: v_lshlrev_b16_e32 v33, 8, v5
+; GFX9-NEXT: v_lshlrev_b16_e32 v48, 8, v5
; GFX9-NEXT: s_waitcnt vmcnt(17)
; GFX9-NEXT: v_lshlrev_b16_e32 v37, 8, v6
; GFX9-NEXT: s_waitcnt vmcnt(16)
@@ -82065,33 +82068,33 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_mov_b32 s7, 0x5040100
; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
; GFX9-NEXT: v_perm_b32 v7, v26, v27, s6
-; GFX9-NEXT: v_perm_b32 v8, v30, v36, s6
-; GFX9-NEXT: v_or_b32_sdwa v9, v47, v45 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
-; GFX9-NEXT: v_perm_b32 v10, v63, v62, s6
+; GFX9-NEXT: v_perm_b32 v8, v30, v33, s6
+; GFX9-NEXT: v_or_b32_sdwa v9, v59, v58 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v10, v34, v36, s6
; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_or_b32_sdwa v11, v55, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v11, v55, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v12, v42, v41, s6
; GFX9-NEXT: v_or_b32_sdwa v13, v43, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
-; GFX9-NEXT: v_perm_b32 v14, v58, v57, s6
-; GFX9-NEXT: v_or_b32_sdwa v15, v34, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v47, v46, s6
+; GFX9-NEXT: v_or_b32_sdwa v15, v56, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: ; implicit-def: $vgpr23
; GFX9-NEXT: ; implicit-def: $vgpr26
; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr30
+; GFX9-NEXT: ; implicit-def: $vgpr33
+; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: ; implicit-def: $vgpr34
; GFX9-NEXT: ; implicit-def: $vgpr36
-; GFX9-NEXT: ; implicit-def: $vgpr47
-; GFX9-NEXT: ; implicit-def: $vgpr63
-; GFX9-NEXT: ; implicit-def: $vgpr62
; GFX9-NEXT: ; implicit-def: $vgpr55
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr41
; GFX9-NEXT: ; implicit-def: $vgpr43
+; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr46
+; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr58
-; GFX9-NEXT: ; implicit-def: $vgpr57
-; GFX9-NEXT: ; implicit-def: $vgpr34
-; GFX9-NEXT: ; implicit-def: $vgpr45
-; GFX9-NEXT: ; implicit-def: $vgpr33
+; GFX9-NEXT: ; implicit-def: $vgpr48
; GFX9-NEXT: ; implicit-def: $vgpr35
; GFX9-NEXT: ; implicit-def: $vgpr39
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -82143,19 +82146,19 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s7
; GFX9-NEXT: v_perm_b32 v7, v28, v29, s6
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s7
-; GFX9-NEXT: v_or_b32_sdwa v8, v59, v56 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v8, v61, v60 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v8, v9, v8, s7
-; GFX9-NEXT: v_or_b32_sdwa v9, v60, v61 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v9, v62, v63 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v9, v9, v10, s7
-; GFX9-NEXT: v_or_b32_sdwa v10, v48, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v10, v31, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v10, v11, v10, s7
; GFX9-NEXT: v_or_b32_sdwa v11, v40, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v11, v11, v12, s7
; GFX9-NEXT: v_or_b32_sdwa v12, v44, v51 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v12, v13, v12, s7
-; GFX9-NEXT: v_or_b32_sdwa v13, v46, v50 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v13, v45, v50 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v13, v13, v14, s7
-; GFX9-NEXT: v_or_b32_sdwa v14, v31, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v14, v57, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v14, v15, v14, s7
; GFX9-NEXT: v_or_b32_sdwa v15, v53, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v15, v15, v16, s7
@@ -82199,16 +82202,16 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
-; GFX9-NEXT: ; implicit-def: $vgpr59
-; GFX9-NEXT: ; implicit-def: $vgpr60
-; GFX9-NEXT: ; implicit-def: $vgpr48
+; GFX9-NEXT: ; implicit-def: $vgpr61
+; GFX9-NEXT: ; implicit-def: $vgpr62
+; GFX9-NEXT: ; implicit-def: $vgpr31
; GFX9-NEXT: ; implicit-def: $vgpr40
; GFX9-NEXT: ; implicit-def: $vgpr44
-; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr31
+; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr57
; GFX9-NEXT: ; implicit-def: $vgpr53
-; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr61
+; GFX9-NEXT: ; implicit-def: $vgpr60
+; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr38
; GFX9-NEXT: ; implicit-def: $vgpr37
; GFX9-NEXT: ; implicit-def: $vgpr51
@@ -82224,19 +82227,18 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v1, 3, v53
; GFX9-NEXT: v_or_b32_sdwa v1, v32, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: s_waitcnt vmcnt(9)
-; GFX9-NEXT: v_add_u16_e32 v2, 3, v31
; GFX9-NEXT: v_add_u16_e32 v15, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v31, 0x300, v1
+; GFX9-NEXT: v_add_u16_e32 v32, 0x300, v1
; GFX9-NEXT: s_waitcnt vmcnt(6)
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v58
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v57
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v47
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v46
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v46
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v2, 3, v57
; GFX9-NEXT: v_or_b32_sdwa v0, v50, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v3, 3, v34
-; GFX9-NEXT: v_add_u16_e32 v34, 0x300, v0
+; GFX9-NEXT: v_or_b32_sdwa v2, v49, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v49, 0x300, v0
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_add_u16_e32 v0, 3, v44
; GFX9-NEXT: v_or_b32_sdwa v0, v51, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -82252,33 +82254,32 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v0, 3, v40
; GFX9-NEXT: v_or_b32_sdwa v0, v37, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v37, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v48
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v31
; GFX9-NEXT: v_or_b32_sdwa v0, v38, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v0
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v0, 3, v55
-; GFX9-NEXT: v_or_b32_sdwa v0, v33, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v33, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v63
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v62
+; GFX9-NEXT: v_or_b32_sdwa v0, v48, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v31, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v34
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v36
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v60
-; GFX9-NEXT: v_or_b32_sdwa v0, v61, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v38, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v59
-; GFX9-NEXT: v_or_b32_sdwa v0, v56, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v62
+; GFX9-NEXT: v_or_b32_sdwa v0, v63, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v34, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v61
+; GFX9-NEXT: v_or_b32_sdwa v0, v60, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v47
-; GFX9-NEXT: v_or_b32_sdwa v0, v45, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_or_b32_sdwa v3, v39, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v39, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v59
+; GFX9-NEXT: v_or_b32_sdwa v0, v58, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v36, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v28
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v29
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v7, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v30
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v36
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v33
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v28, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v24
@@ -82303,21 +82304,22 @@ define <32 x half> @bitcast_v64i8_to_v32f16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v4, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v18
-; GFX9-NEXT: v_add_u16_e32 v32, 0x300, v3
-; GFX9-NEXT: v_or_b32_sdwa v2, v49, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v3, 3, v56
+; GFX9-NEXT: v_or_b32_sdwa v3, v39, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v39, 0x300, v3
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v2
; GFX9-NEXT: s_mov_b32 s6, 0x5040100
; GFX9-NEXT: v_perm_b32 v5, v20, v5, s6
; GFX9-NEXT: v_perm_b32 v6, v24, v6, s6
; GFX9-NEXT: v_perm_b32 v7, v28, v7, s6
-; GFX9-NEXT: v_perm_b32 v8, v39, v8, s6
-; GFX9-NEXT: v_perm_b32 v9, v38, v9, s6
-; GFX9-NEXT: v_perm_b32 v10, v33, v10, s6
+; GFX9-NEXT: v_perm_b32 v8, v36, v8, s6
+; GFX9-NEXT: v_perm_b32 v9, v34, v9, s6
+; GFX9-NEXT: v_perm_b32 v10, v31, v10, s6
; GFX9-NEXT: v_perm_b32 v11, v37, v11, s6
; GFX9-NEXT: v_perm_b32 v12, v35, v12, s6
-; GFX9-NEXT: v_perm_b32 v13, v34, v13, s6
-; GFX9-NEXT: v_perm_b32 v14, v32, v14, s6
-; GFX9-NEXT: v_perm_b32 v15, v31, v15, s6
+; GFX9-NEXT: v_perm_b32 v13, v49, v13, s6
+; GFX9-NEXT: v_perm_b32 v14, v39, v14, s6
+; GFX9-NEXT: v_perm_b32 v15, v32, v15, s6
; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -83944,10 +83946,11 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_readfirstlane_b32 s7, v21
; VI-NEXT: v_readfirstlane_b32 s9, v20
; VI-NEXT: v_readfirstlane_b32 s6, v19
-; VI-NEXT: v_readfirstlane_b32 s44, v18
-; VI-NEXT: v_readfirstlane_b32 s42, v17
-; VI-NEXT: v_readfirstlane_b32 s43, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; VI-NEXT: v_readfirstlane_b32 s56, v18
+; VI-NEXT: v_readfirstlane_b32 s43, v17
+; VI-NEXT: v_readfirstlane_b32 s46, v16
+; VI-NEXT: v_readfirstlane_b32 s45, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:4
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32
@@ -83967,24 +83970,23 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:12
-; VI-NEXT: v_readfirstlane_b32 s74, v15
-; VI-NEXT: v_readfirstlane_b32 s76, v14
-; VI-NEXT: v_readfirstlane_b32 s72, v13
-; VI-NEXT: v_readfirstlane_b32 s75, v12
-; VI-NEXT: v_readfirstlane_b32 s62, v11
-; VI-NEXT: v_readfirstlane_b32 s73, v10
-; VI-NEXT: v_readfirstlane_b32 s60, v9
-; VI-NEXT: v_readfirstlane_b32 s63, v8
-; VI-NEXT: v_readfirstlane_b32 s58, v7
-; VI-NEXT: v_readfirstlane_b32 s61, v6
-; VI-NEXT: v_readfirstlane_b32 s47, v5
-; VI-NEXT: v_readfirstlane_b32 s59, v4
-; VI-NEXT: v_readfirstlane_b32 s46, v3
-; VI-NEXT: v_readfirstlane_b32 s56, v2
-; VI-NEXT: v_readfirstlane_b32 s45, v1
-; VI-NEXT: v_readfirstlane_b32 s57, v0
+; VI-NEXT: v_readfirstlane_b32 s44, v15
+; VI-NEXT: v_readfirstlane_b32 s47, v14
+; VI-NEXT: v_readfirstlane_b32 s42, v13
+; VI-NEXT: v_readfirstlane_b32 s74, v11
+; VI-NEXT: v_readfirstlane_b32 s76, v10
+; VI-NEXT: v_readfirstlane_b32 s72, v9
+; VI-NEXT: v_readfirstlane_b32 s75, v8
+; VI-NEXT: v_readfirstlane_b32 s62, v7
+; VI-NEXT: v_readfirstlane_b32 s73, v6
+; VI-NEXT: v_readfirstlane_b32 s59, v5
+; VI-NEXT: v_readfirstlane_b32 s63, v4
+; VI-NEXT: v_readfirstlane_b32 s58, v3
+; VI-NEXT: v_readfirstlane_b32 s60, v2
+; VI-NEXT: v_readfirstlane_b32 s57, v1
+; VI-NEXT: v_readfirstlane_b32 s61, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB107_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -84006,41 +84008,41 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: v_perm_b32 v3, s26, v3, v11
; VI-NEXT: v_perm_b32 v2, s24, v2, v11
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; VI-NEXT: v_mov_b32_e32 v4, s45
+; VI-NEXT: v_mov_b32_e32 v4, s57
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s57, v4, v11
+; VI-NEXT: v_perm_b32 v4, s61, v4, v11
; VI-NEXT: v_perm_b32 v3, s28, v3, v11
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_mov_b32_e32 v5, s47
+; VI-NEXT: v_mov_b32_e32 v5, s59
; VI-NEXT: v_or_b32_e32 v3, v3, v4
-; VI-NEXT: v_mov_b32_e32 v4, s46
-; VI-NEXT: v_perm_b32 v5, s59, v5, v11
-; VI-NEXT: v_perm_b32 v4, s56, v4, v11
+; VI-NEXT: v_mov_b32_e32 v4, s58
+; VI-NEXT: v_perm_b32 v5, s63, v5, v11
+; VI-NEXT: v_perm_b32 v4, s60, v4, v11
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; VI-NEXT: v_mov_b32_e32 v6, s60
+; VI-NEXT: v_mov_b32_e32 v6, s72
; VI-NEXT: v_or_b32_e32 v4, v4, v5
-; VI-NEXT: v_mov_b32_e32 v5, s58
-; VI-NEXT: v_perm_b32 v6, s63, v6, v11
-; VI-NEXT: v_perm_b32 v5, s61, v5, v11
+; VI-NEXT: v_mov_b32_e32 v5, s62
+; VI-NEXT: v_perm_b32 v6, s75, v6, v11
+; VI-NEXT: v_perm_b32 v5, s73, v5, v11
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; VI-NEXT: v_mov_b32_e32 v7, s72
+; VI-NEXT: v_mov_b32_e32 v7, s42
; VI-NEXT: v_or_b32_e32 v5, v5, v6
-; VI-NEXT: v_mov_b32_e32 v6, s62
-; VI-NEXT: v_perm_b32 v7, s75, v7, v11
-; VI-NEXT: v_perm_b32 v6, s73, v6, v11
+; VI-NEXT: v_mov_b32_e32 v6, s74
+; VI-NEXT: v_perm_b32 v7, s45, v7, v11
+; VI-NEXT: v_perm_b32 v6, s76, v6, v11
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; VI-NEXT: v_mov_b32_e32 v8, s42
+; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
-; VI-NEXT: v_mov_b32_e32 v7, s74
-; VI-NEXT: v_perm_b32 v8, s43, v8, v11
-; VI-NEXT: v_perm_b32 v7, s76, v7, v11
+; VI-NEXT: v_mov_b32_e32 v7, s44
+; VI-NEXT: v_perm_b32 v8, s46, v8, v11
+; VI-NEXT: v_perm_b32 v7, s47, v7, v11
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s7
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s6
; VI-NEXT: v_perm_b32 v9, s9, v9, v11
-; VI-NEXT: v_perm_b32 v8, s44, v8, v11
+; VI-NEXT: v_perm_b32 v8, s56, v8, v11
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s10
; VI-NEXT: v_or_b32_e32 v8, v8, v9
@@ -84131,36 +84133,36 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; VI-NEXT: s_add_i32 s9, s9, 3
; VI-NEXT: v_mov_b32_e32 v1, s7
; VI-NEXT: v_perm_b32 v22, s9, v1, v0
-; VI-NEXT: s_add_i32 s44, s44, 3
+; VI-NEXT: s_add_i32 s56, s56, 3
; VI-NEXT: v_mov_b32_e32 v1, s6
-; VI-NEXT: v_perm_b32 v23, s44, v1, v0
-; VI-NEXT: s_add_i32 s43, s43, 3
+; VI-NEXT: v_perm_b32 v23, s56, v1, v0
+; VI-NEXT: s_add_i32 s46, s46, 3
+; VI-NEXT: v_mov_b32_e32 v1, s43
+; VI-NEXT: v_perm_b32 v7, s46, v1, v0
+; VI-NEXT: s_add_i32 s47, s47, 3
+; VI-NEXT: v_mov_b32_e32 v1, s44
+; VI-NEXT: v_perm_b32 v24, s47, v1, v0
+; VI-NEXT: s_add_i32 s45, s45, 3
; VI-NEXT: v_mov_b32_e32 v1, s42
-; VI-NEXT: v_perm_b32 v7, s43, v1, v0
+; VI-NEXT: v_perm_b32 v6, s45, v1, v0
; VI-NEXT: s_add_i32 s76, s76, 3
; VI-NEXT: v_mov_b32_e32 v1, s74
-; VI-NEXT: v_perm_b32 v24, s76, v1, v0
+; VI-NEXT: v_perm_b32 v25, s76, v1, v0
; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: v_mov_b32_e32 v1, s72
-; VI-NEXT: v_perm_b32 v6, s75, v1, v0
+; VI-NEXT: v_perm_b32 v5, s75, v1, v0
; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: v_mov_b32_e32 v1, s62
-; VI-NEXT: v_perm_b32 v25, s73, v1, v0
+; VI-NEXT: v_perm_b32 v26, s73, v1, v0
; VI-NEXT: s_add_i32 s63, s63, 3
-; VI-NEXT: v_mov_b32_e32 v1, s60
-; VI-NEXT: v_perm_b32 v5, s63, v1, v0
-; VI-NEXT: s_add_i32 s61, s61, 3
+; VI-NEXT: v_mov_b32_e32 v1, s59
+; VI-NEXT: v_perm_b32 v4, s63, v1, v0
+; VI-NEXT: s_add_i32 s60, s60, 3
; VI-NEXT: v_mov_b32_e32 v1, s58
-; VI-NEXT: v_perm_b32 v26, s61, v1, v0
-; VI-NEXT: s_add_i32 s59, s59, 3
-; VI-NEXT: v_mov_b32_e32 v1, s47
-; VI-NEXT: v_perm_b32 v4, s59, v1, v0
-; VI-NEXT: s_add_i32 s56, s56, 3
-; VI-NEXT: v_mov_b32_e32 v1, s46
-; VI-NEXT: v_perm_b32 v27, s56, v1, v0
-; VI-NEXT: s_add_i32 s57, s57, 3
-; VI-NEXT: v_mov_b32_e32 v1, s45
-; VI-NEXT: v_perm_b32 v3, s57, v1, v0
+; VI-NEXT: v_perm_b32 v27, s60, v1, v0
+; VI-NEXT: s_add_i32 s61, s61, 3
+; VI-NEXT: v_mov_b32_e32 v1, s57
+; VI-NEXT: v_perm_b32 v3, s61, v1, v0
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: v_mov_b32_e32 v1, s29
; VI-NEXT: v_perm_b32 v28, s28, v1, v0
@@ -84256,9 +84258,9 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9: ; %bb.0:
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX9-NEXT: v_readfirstlane_b32 s72, v30
-; GFX9-NEXT: v_readfirstlane_b32 s47, v29
+; GFX9-NEXT: v_readfirstlane_b32 s56, v29
; GFX9-NEXT: v_readfirstlane_b32 s57, v28
-; GFX9-NEXT: v_readfirstlane_b32 s60, v27
+; GFX9-NEXT: v_readfirstlane_b32 s61, v27
; GFX9-NEXT: v_readfirstlane_b32 s62, v26
; GFX9-NEXT: v_readfirstlane_b32 s7, v25
; GFX9-NEXT: v_readfirstlane_b32 s46, v24
@@ -84270,7 +84272,8 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_readfirstlane_b32 s10, v18
; GFX9-NEXT: v_readfirstlane_b32 s11, v17
; GFX9-NEXT: v_readfirstlane_b32 s12, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; GFX9-NEXT: v_readfirstlane_b32 s15, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:4
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32
@@ -84291,23 +84294,22 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:12
; GFX9-NEXT: v_readfirstlane_b32 s13, v15
-; GFX9-NEXT: v_readfirstlane_b32 s76, v14
+; GFX9-NEXT: v_readfirstlane_b32 s73, v14
; GFX9-NEXT: v_readfirstlane_b32 s14, v13
-; GFX9-NEXT: v_readfirstlane_b32 s15, v12
; GFX9-NEXT: v_readfirstlane_b32 s40, v11
-; GFX9-NEXT: v_readfirstlane_b32 s75, v10
+; GFX9-NEXT: v_readfirstlane_b32 s76, v10
; GFX9-NEXT: v_readfirstlane_b32 s41, v9
; GFX9-NEXT: v_readfirstlane_b32 s42, v8
; GFX9-NEXT: v_readfirstlane_b32 s43, v7
-; GFX9-NEXT: v_readfirstlane_b32 s74, v6
+; GFX9-NEXT: v_readfirstlane_b32 s75, v6
; GFX9-NEXT: v_readfirstlane_b32 s44, v5
; GFX9-NEXT: v_readfirstlane_b32 s45, v4
-; GFX9-NEXT: v_readfirstlane_b32 s56, v3
-; GFX9-NEXT: v_readfirstlane_b32 s73, v2
-; GFX9-NEXT: v_readfirstlane_b32 s61, v1
+; GFX9-NEXT: v_readfirstlane_b32 s47, v3
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s60, v1
; GFX9-NEXT: v_readfirstlane_b32 s63, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB107_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -84329,7 +84331,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v11
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; GFX9-NEXT: v_mov_b32_e32 v4, s61
+; GFX9-NEXT: v_mov_b32_e32 v4, s60
; GFX9-NEXT: v_or_b32_sdwa v2, v3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s63, v4, v11
@@ -84337,27 +84339,27 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s44
; GFX9-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_mov_b32_e32 v4, s56
+; GFX9-NEXT: v_mov_b32_e32 v4, s47
; GFX9-NEXT: v_perm_b32 v5, s45, v5, v11
-; GFX9-NEXT: v_perm_b32 v4, s73, v4, v11
+; GFX9-NEXT: v_perm_b32 v4, s74, v4, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s41
; GFX9-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v5, s43
; GFX9-NEXT: v_perm_b32 v6, s42, v6, v11
-; GFX9-NEXT: v_perm_b32 v5, s74, v5, v11
+; GFX9-NEXT: v_perm_b32 v5, s75, v5, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s14
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v6, s40
; GFX9-NEXT: v_perm_b32 v7, s15, v7, v11
-; GFX9-NEXT: v_perm_b32 v6, s75, v6, v11
+; GFX9-NEXT: v_perm_b32 v6, s76, v6, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; GFX9-NEXT: v_mov_b32_e32 v8, s11
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v7, s13
; GFX9-NEXT: v_perm_b32 v8, s12, v8, v11
-; GFX9-NEXT: v_perm_b32 v7, s76, v7, v11
+; GFX9-NEXT: v_perm_b32 v7, s73, v7, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v10, 0x6050400
; GFX9-NEXT: v_mov_b32_e32 v9, s8
@@ -84375,8 +84377,8 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_or_b32_sdwa v9, v12, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v12, s62
; GFX9-NEXT: v_mov_b32_e32 v13, s57
-; GFX9-NEXT: v_perm_b32 v12, s60, v12, v10
-; GFX9-NEXT: v_perm_b32 v10, s47, v13, v10
+; GFX9-NEXT: v_perm_b32 v12, s61, v12, v10
+; GFX9-NEXT: v_perm_b32 v10, s56, v13, v10
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: s_mov_b32 s4, 0x6050400
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -84441,10 +84443,10 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_mov_b32_e32 v19, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v16, v0, v16, s4
; GFX9-NEXT: s_add_i32 s62, s62, 3
-; GFX9-NEXT: v_mov_b32_e32 v0, s60
+; GFX9-NEXT: v_mov_b32_e32 v0, s61
; GFX9-NEXT: v_perm_b32 v17, s62, v0, v19
; GFX9-NEXT: s_add_i32 s57, s57, 3
-; GFX9-NEXT: v_mov_b32_e32 v0, s47
+; GFX9-NEXT: v_mov_b32_e32 v0, s56
; GFX9-NEXT: v_perm_b32 v20, s57, v0, v19
; GFX9-NEXT: s_add_i32 s59, s59, 3
; GFX9-NEXT: v_mov_b32_e32 v0, s58
@@ -84480,44 +84482,44 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; GFX9-NEXT: v_or_b32_sdwa v2, v3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: s_add_i32 s63, s63, 3
-; GFX9-NEXT: v_mov_b32_e32 v4, s61
+; GFX9-NEXT: v_mov_b32_e32 v4, s60
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v19
; GFX9-NEXT: v_perm_b32 v4, s63, v4, v19
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s73, s73, 3
+; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_mov_b32_e32 v4, s56
+; GFX9-NEXT: v_mov_b32_e32 v4, s47
; GFX9-NEXT: s_add_i32 s45, s45, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s44
-; GFX9-NEXT: v_perm_b32 v4, s73, v4, v19
+; GFX9-NEXT: v_perm_b32 v4, s74, v4, v19
; GFX9-NEXT: v_perm_b32 v5, s45, v5, v19
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s74, s74, 3
+; GFX9-NEXT: s_add_i32 s75, s75, 3
; GFX9-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v5, s43
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s41
-; GFX9-NEXT: v_perm_b32 v5, s74, v5, v19
+; GFX9-NEXT: v_perm_b32 v5, s75, v5, v19
; GFX9-NEXT: v_perm_b32 v6, s42, v6, v19
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s75, s75, 3
+; GFX9-NEXT: s_add_i32 s76, s76, 3
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v6, s40
; GFX9-NEXT: s_add_i32 s15, s15, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s14
-; GFX9-NEXT: v_perm_b32 v6, s75, v6, v19
+; GFX9-NEXT: v_perm_b32 v6, s76, v6, v19
; GFX9-NEXT: v_perm_b32 v7, s15, v7, v19
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s76, s76, 3
+; GFX9-NEXT: s_add_i32 s73, s73, 3
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v7, s13
; GFX9-NEXT: s_add_i32 s12, s12, 3
; GFX9-NEXT: v_mov_b32_e32 v22, s11
-; GFX9-NEXT: v_perm_b32 v7, s76, v7, v19
+; GFX9-NEXT: v_perm_b32 v7, s73, v7, v19
; GFX9-NEXT: v_perm_b32 v22, s12, v22, v19
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -91421,172 +91423,157 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:48
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:64
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:80
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:88
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:64
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:80
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:88
; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:96
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:112
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:120
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:128
-; SI-NEXT: buffer_load_dword v57, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:104
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:36
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128
+; SI-NEXT: buffer_load_dword v47, off, s[0:3], s32 offset:108
+; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:92
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:20
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:12
; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v3
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v3, 8, v5
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v7
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v11
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:288 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v15
-; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:4
-; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
; SI-NEXT: ; implicit-def: $vgpr11
; SI-NEXT: ; kill: killed $vgpr11
; SI-NEXT: ; implicit-def: $vgpr11
; SI-NEXT: ; kill: killed $vgpr11
; SI-NEXT: ; implicit-def: $vgpr11
-; SI-NEXT: v_lshlrev_b32_e32 v38, 8, v13
-; SI-NEXT: v_lshlrev_b32_e32 v36, 24, v19
-; SI-NEXT: v_lshlrev_b32_e32 v52, 8, v21
-; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v23
-; SI-NEXT: v_lshlrev_b32_e32 v50, 24, v27
-; SI-NEXT: v_lshlrev_b32_e32 v44, 8, v29
; SI-NEXT: ; kill: killed $vgpr11
; SI-NEXT: ; implicit-def: $vgpr11
; SI-NEXT: ; kill: killed $vgpr11
+; SI-NEXT: ; implicit-def: $vgpr11
+; SI-NEXT: ; kill: killed $vgpr11
+; SI-NEXT: ; implicit-def: $vgpr11
+; SI-NEXT: v_lshlrev_b32_e32 v50, 8, v13
+; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v15
+; SI-NEXT: v_lshlrev_b32_e32 v48, 24, v19
+; SI-NEXT: v_lshlrev_b32_e32 v42, 8, v21
+; SI-NEXT: v_lshlrev_b32_e32 v53, 24, v23
+; SI-NEXT: v_lshlrev_b32_e32 v55, 24, v27
+; SI-NEXT: v_lshlrev_b32_e32 v59, 8, v29
+; SI-NEXT: ; kill: killed $vgpr11
+; SI-NEXT: ; implicit-def: $vgpr11
+; SI-NEXT: ; kill: killed $vgpr11
+; SI-NEXT: ; implicit-def: $vgpr21
; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr29
-; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr51
-; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: ; implicit-def: $vgpr54
; SI-NEXT: ; implicit-def: $vgpr40
; SI-NEXT: ; implicit-def: $vgpr41
+; SI-NEXT: ; implicit-def: $vgpr43
; SI-NEXT: ; implicit-def: $vgpr45
-; SI-NEXT: ; implicit-def: $vgpr46
-; SI-NEXT: ; implicit-def: $vgpr47
-; SI-NEXT: ; implicit-def: $vgpr58
+; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: ; implicit-def: $vgpr57
; SI-NEXT: ; implicit-def: $vgpr60
-; SI-NEXT: ; implicit-def: $vgpr61
-; SI-NEXT: ; implicit-def: $vgpr63
+; SI-NEXT: ; implicit-def: $vgpr62
+; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr11
-; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr13
+; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr15
-; SI-NEXT: ; implicit-def: $vgpr33
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr19
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; SI-NEXT: v_lshlrev_b32_e32 v55, 24, v1
-; SI-NEXT: v_lshlrev_b32_e32 v42, 24, v2
-; SI-NEXT: v_lshlrev_b32_e32 v62, 8, v4
-; SI-NEXT: v_lshlrev_b32_e32 v56, 24, v6
-; SI-NEXT: v_lshlrev_b32_e32 v59, 24, v8
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v12
-; SI-NEXT: v_lshlrev_b32_e32 v0, 24, v16
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v3, 8, v17
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:84
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:72
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:40
-; SI-NEXT: v_lshlrev_b32_e32 v28, 24, v20
-; SI-NEXT: v_lshlrev_b32_e32 v19, 8, v24
-; SI-NEXT: v_lshlrev_b32_e32 v21, 24, v26
-; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: v_lshlrev_b32_e32 v46, 24, v1
+; SI-NEXT: v_lshlrev_b32_e32 v58, 24, v2
+; SI-NEXT: v_lshlrev_b32_e32 v26, 8, v4
+; SI-NEXT: v_lshlrev_b32_e32 v2, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v1, 24, v8
+; SI-NEXT: v_lshlrev_b32_e32 v17, 8, v9
+; SI-NEXT: v_lshlrev_b32_e32 v4, 24, v10
+; SI-NEXT: v_lshlrev_b32_e32 v25, 24, v12
+; SI-NEXT: v_lshlrev_b32_e32 v6, 8, v16
+; SI-NEXT: v_lshlrev_b32_e32 v20, 24, v20
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v22
+; SI-NEXT: s_waitcnt vmcnt(13)
+; SI-NEXT: v_lshlrev_b32_e32 v34, 8, v24
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_lshlrev_b32_e32 v35, 24, v28
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:84
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:68
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:60
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:40
+; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: ; implicit-def: $vgpr16
-; SI-NEXT: ; implicit-def: $vgpr20
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:20
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:12
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:8
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:8
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:4
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:116
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v9
-; SI-NEXT: v_lshlrev_b32_e32 v9, 24, v22
-; SI-NEXT: ; implicit-def: $vgpr22
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:124
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB110_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_lshlrev_b32_e32 v1, 24, v1
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v32, v59, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v5
-; SI-NEXT: v_or_b32_e32 v4, v4, v14
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v12
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v26, v6, v4
-; SI-NEXT: ; implicit-def: $vgpr4
-; SI-NEXT: ; implicit-def: $vgpr5
-; SI-NEXT: ; implicit-def: $vgpr12
-; SI-NEXT: ; implicit-def: $vgpr59
-; SI-NEXT: ; implicit-def: $vgpr14
-; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: v_or_b32_e32 v0, v0, v17
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v5
+; SI-NEXT: ; implicit-def: $vgpr17
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v13
; SI-NEXT: v_or_b32_e32 v11, v13, v11
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v11, v13, v11
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_or_b32_e32 v11, v11, v13
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(2)
@@ -91594,143 +91581,137 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v13
-; SI-NEXT: v_or_b32_e32 v23, v13, v11
+; SI-NEXT: v_or_b32_e32 v11, v13, v11
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v27, v13, v11
+; SI-NEXT: v_or_b32_e32 v11, v13, v11
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v11, v11, v38
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v11
+; SI-NEXT: v_or_b32_e32 v11, v11, v50
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v13
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v49, v13, v11
+; SI-NEXT: v_or_b32_e32 v21, v38, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v13
-; SI-NEXT: v_or_b32_e32 v51, v13, v11
+; SI-NEXT: v_or_b32_e32 v23, v13, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v53, v36, v11
+; SI-NEXT: v_or_b32_e32 v27, v48, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v13
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v11, v11, v52
-; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v11
+; SI-NEXT: v_or_b32_e32 v11, v11, v42
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr52
+; SI-NEXT: ; implicit-def: $vgpr42
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v40, v48, v11
+; SI-NEXT: v_or_b32_e32 v51, v53, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr48
+; SI-NEXT: ; implicit-def: $vgpr53
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v41, v13, v11
+; SI-NEXT: v_or_b32_e32 v54, v13, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v36
+; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v45, v50, v11
+; SI-NEXT: v_or_b32_e32 v40, v55, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v13
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr55
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v11, v11, v44
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v11
+; SI-NEXT: v_or_b32_e32 v11, v11, v59
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v11
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr59
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v47, v55, v11
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v35
+; SI-NEXT: v_or_b32_e32 v43, v46, v11
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v39
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v58, v13, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
-; SI-NEXT: v_lshlrev_b32_e32 v13, 24, v17
-; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr17
-; SI-NEXT: ; implicit-def: $vgpr55
+; SI-NEXT: v_or_b32_e32 v45, v13, v11
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr46
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v60, v42, v11
+; SI-NEXT: v_or_b32_e32 v56, v58, v11
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v49
+; SI-NEXT: v_or_b32_e32 v11, v11, v26
+; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v11
; SI-NEXT: v_and_b32_e32 v11, 0xff, v37
-; SI-NEXT: v_or_b32_e32 v11, v11, v62
-; SI-NEXT: v_lshlrev_b32_e32 v61, 16, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
-; SI-NEXT: ; implicit-def: $vgpr37
-; SI-NEXT: ; implicit-def: $vgpr42
-; SI-NEXT: ; implicit-def: $vgpr62
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v63, v56, v11
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v39
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v11, v13, v11
-; SI-NEXT: v_or_b32_e32 v13, v1, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v1
-; SI-NEXT: v_or_b32_e32 v30, v0, v1
+; SI-NEXT: v_or_b32_e32 v60, v2, v11
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v52
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v3
+; SI-NEXT: v_or_b32_e32 v62, v5, v2
+; SI-NEXT: v_and_b32_e32 v2, 0xff, v28
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
+; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
+; SI-NEXT: v_or_b32_e32 v13, v4, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v0, v0, v3
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v43
+; SI-NEXT: v_or_b32_e32 v31, v1, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v24, v28, v0
+; SI-NEXT: v_lshlrev_b32_e32 v1, 24, v9
+; SI-NEXT: v_or_b32_e32 v22, v1, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v10
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_lshlrev_b32_e32 v1, 24, v31
-; SI-NEXT: v_or_b32_e32 v16, v1, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v20, v9, v0
-; SI-NEXT: v_and_b32_e32 v0, 0xff, v25
-; SI-NEXT: v_or_b32_e32 v0, v0, v19
+; SI-NEXT: v_or_b32_e32 v32, v25, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v12
+; SI-NEXT: v_or_b32_e32 v0, v0, v6
; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v44
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
+; SI-NEXT: v_or_b32_e32 v30, v20, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v63
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
+; SI-NEXT: v_lshlrev_b32_e32 v1, 24, v61
+; SI-NEXT: v_or_b32_e32 v15, v1, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v47
+; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
+; SI-NEXT: v_or_b32_e32 v19, v7, v0
; SI-NEXT: v_and_b32_e32 v0, 0xff, v18
+; SI-NEXT: v_or_b32_e32 v0, v0, v34
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v14
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
-; SI-NEXT: v_or_b32_e32 v34, v21, v0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; kill: killed $vgpr0
+; SI-NEXT: v_or_b32_e32 v16, v35, v0
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
@@ -91766,281 +91747,298 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
+; SI-NEXT: ; kill: killed $vgpr1
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
+; SI-NEXT: ; kill: killed $vgpr1
+; SI-NEXT: ; implicit-def: $vgpr1
; SI-NEXT: ; kill: killed $vgpr0
; SI-NEXT: ; implicit-def: $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr2
+; SI-NEXT: ; kill: killed $vgpr1
; SI-NEXT: ; implicit-def: $vgpr1
-; SI-NEXT: ; implicit-def: $vgpr7
-; SI-NEXT: ; implicit-def: $vgpr8
-; SI-NEXT: ; implicit-def: $vgpr43
-; SI-NEXT: ; implicit-def: $vgpr10
-; SI-NEXT: ; implicit-def: $vgpr31
-; SI-NEXT: ; implicit-def: $vgpr57
-; SI-NEXT: ; implicit-def: $vgpr25
-; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; kill: killed $vgpr0
-; SI-NEXT: ; implicit-def: $vgpr56
+; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr37
+; SI-NEXT: ; implicit-def: $vgpr52
+; SI-NEXT: ; implicit-def: $vgpr5
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr0
; SI-NEXT: ; implicit-def: $vgpr3
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr8
; SI-NEXT: ; implicit-def: $vgpr9
-; SI-NEXT: ; implicit-def: $vgpr19
-; SI-NEXT: ; implicit-def: $vgpr21
+; SI-NEXT: ; implicit-def: $vgpr10
+; SI-NEXT: ; implicit-def: $vgpr12
+; SI-NEXT: ; implicit-def: $vgpr44
+; SI-NEXT: ; implicit-def: $vgpr63
+; SI-NEXT: ; implicit-def: $vgpr61
+; SI-NEXT: ; implicit-def: $vgpr47
+; SI-NEXT: ; implicit-def: $vgpr18
+; SI-NEXT: ; implicit-def: $vgpr14
+; SI-NEXT: ; kill: killed $vgpr1
+; SI-NEXT: ; implicit-def: $vgpr58
+; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr2
+; SI-NEXT: ; implicit-def: $vgpr1
+; SI-NEXT: ; implicit-def: $vgpr4
+; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr6
+; SI-NEXT: ; implicit-def: $vgpr20
+; SI-NEXT: ; implicit-def: $vgpr7
+; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: .LBB110_2: ; %Flow
; SI-NEXT: s_andn2_saveexec_b64 s[4:5], s[4:5]
; SI-NEXT: s_cbranch_execz .LBB110_4
; SI-NEXT: ; %bb.3: ; %cmp.true
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v2
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v1, 8, v1
-; SI-NEXT: v_or_b32_e32 v1, v1, v2
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v7
+; SI-NEXT: s_waitcnt vmcnt(7)
+; SI-NEXT: v_add_i32_e32 v0, vcc, 3, v0
+; SI-NEXT: v_and_b32_e32 v0, 0xff, v0
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v3
; SI-NEXT: s_movk_i32 s6, 0x300
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v0, v0, v2
-; SI-NEXT: v_or_b32_e32 v0, v0, v1
-; SI-NEXT: v_add_i32_e32 v1, vcc, 3, v5
-; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v12
-; SI-NEXT: v_or_b32_e32 v1, v14, v1
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_add_i32_e32 v1, vcc, s6, v1
-; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
-; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_or_b32_e32 v2, v6, v2
-; SI-NEXT: v_or_b32_e32 v1, v2, v1
-; SI-NEXT: v_add_i32_e32 v2, vcc, 3, v39
-; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v17
-; SI-NEXT: v_or_b32_e32 v2, v5, v2
-; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
+; SI-NEXT: v_or_b32_e32 v0, v17, v0
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: v_add_i32_e32 v0, vcc, s6, v0
+; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
+; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
+; SI-NEXT: v_or_b32_e32 v3, v4, v3
+; SI-NEXT: v_or_b32_e32 v0, v3, v0
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v52
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: s_waitcnt vmcnt(5)
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v4, v3
+; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v28
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_add_i32_e32 v2, vcc, s6, v2
+; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_or_b32_e32 v4, v59, v4
-; SI-NEXT: v_or_b32_e32 v2, v4, v2
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: v_or_b32_e32 v1, v1, v4
+; SI-NEXT: v_or_b32_e32 v1, v1, v3
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v49
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v37
+; SI-NEXT: v_or_b32_e32 v3, v26, v3
; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_or_b32_e32 v4, v62, v4
-; SI-NEXT: v_add_i32_e32 v4, vcc, s6, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v3, v3, v8
-; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v43
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: v_or_b32_e32 v2, v2, v4
+; SI-NEXT: v_or_b32_e32 v2, v2, v3
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_i32_e32 v3, vcc, 3, v39
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v36
+; SI-NEXT: v_or_b32_e32 v3, v4, v3
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v3, vcc, s6, v3
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v8, v28, v8
-; SI-NEXT: v_or_b32_e32 v3, v8, v3
-; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v25
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
+; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
+; SI-NEXT: v_lshlrev_b32_e32 v9, 8, v9
+; SI-NEXT: v_or_b32_e32 v8, v9, v8
+; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v10
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_add_i32_e32 v8, vcc, s6, v8
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
+; SI-NEXT: v_or_b32_e32 v9, v25, v9
+; SI-NEXT: v_or_b32_e32 v8, v9, v8
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v18
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v18
-; SI-NEXT: v_or_b32_e32 v11, v19, v11
+; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v14
+; SI-NEXT: v_or_b32_e32 v11, v34, v11
; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_add_i32_e32 v11, vcc, 0x300, v11
-; SI-NEXT: v_or_b32_e32 v13, v21, v13
+; SI-NEXT: v_or_b32_e32 v13, v35, v13
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_add_i32_e32 v10, vcc, 3, v10
; SI-NEXT: v_or_b32_e32 v11, v13, v11
-; SI-NEXT: v_and_b32_e32 v10, 0xff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v13, 8, v31
-; SI-NEXT: v_or_b32_e32 v13, v13, v10
-; SI-NEXT: v_add_i32_e32 v10, vcc, 0x3000000, v11
+; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v63
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
+; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v61
+; SI-NEXT: v_or_b32_e32 v13, v14, v13
+; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v11
; SI-NEXT: v_add_i32_e32 v11, vcc, s6, v13
-; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v57
+; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v47
; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_or_b32_e32 v9, v9, v13
-; SI-NEXT: v_or_b32_e32 v9, v9, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v9, vcc, 0x3000000, v9
-; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v3
+; SI-NEXT: v_or_b32_e32 v7, v7, v13
+; SI-NEXT: v_or_b32_e32 v7, v7, v11
+; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v12
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
+; SI-NEXT: v_or_b32_e32 v6, v6, v11
+; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v44
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
+; SI-NEXT: v_add_i32_e32 v6, vcc, s6, v6
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
+; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
+; SI-NEXT: v_or_b32_e32 v11, v20, v11
+; SI-NEXT: v_or_b32_e32 v6, v11, v6
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v7, vcc, 0x3000000, v7
+; SI-NEXT: v_add_i32_e32 v6, vcc, 0x3000000, v6
+; SI-NEXT: v_add_i32_e32 v8, vcc, 0x3000000, v8
; SI-NEXT: v_add_i32_e32 v0, vcc, 0x3000000, v0
; SI-NEXT: v_add_i32_e32 v1, vcc, 0x3000000, v1
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3000000, v2
-; SI-NEXT: v_and_b32_e32 v32, 0xffff0000, v2
-; SI-NEXT: v_and_b32_e32 v26, 0xffff0000, v1
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v1
-; SI-NEXT: v_and_b32_e32 v30, 0xffff0000, v0
-; SI-NEXT: v_and_b32_e32 v24, 0xffff0000, v3
+; SI-NEXT: v_and_b32_e32 v60, 0xffff0000, v2
+; SI-NEXT: v_lshlrev_b32_e32 v57, 16, v2
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
+; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
+; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_or_b32_e32 v4, v58, v4
+; SI-NEXT: v_or_b32_e32 v3, v4, v3
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(6)
; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v5
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; SI-NEXT: v_or_b32_e32 v5, v56, v5
+; SI-NEXT: v_or_b32_e32 v5, v46, v5
+; SI-NEXT: v_add_i32_e32 v3, vcc, 0x3000000, v3
+; SI-NEXT: v_and_b32_e32 v56, 0xffff0000, v3
+; SI-NEXT: v_lshlrev_b32_e32 v45, 16, v3
+; SI-NEXT: v_and_b32_e32 v31, 0xffff0000, v1
+; SI-NEXT: v_lshlrev_b32_e32 v62, 16, v1
+; SI-NEXT: v_and_b32_e32 v32, 0xffff0000, v8
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v8
+; SI-NEXT: v_and_b32_e32 v30, 0xffff0000, v6
+; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v6
+; SI-NEXT: v_and_b32_e32 v19, 0xffff0000, v7
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v14
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:308 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(8)
+; SI-NEXT: v_lshlrev_b32_e32 v9, 8, v9
+; SI-NEXT: s_waitcnt vmcnt(7)
+; SI-NEXT: v_add_i32_e32 v10, vcc, 3, v10
+; SI-NEXT: v_and_b32_e32 v10, 0xff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
+; SI-NEXT: v_or_b32_e32 v10, v53, v10
+; SI-NEXT: s_waitcnt vmcnt(6)
+; SI-NEXT: v_lshlrev_b32_e32 v11, 8, v11
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_add_i32_e32 v12, vcc, 3, v12
+; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_or_b32_e32 v12, v38, v12
+; SI-NEXT: v_lshlrev_b32_e32 v13, 8, v13
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_add_i32_e32 v4, vcc, 3, v4
+; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
+; SI-NEXT: v_or_b32_e32 v4, v59, v4
+; SI-NEXT: v_add_i32_e32 v4, vcc, s6, v4
+; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
; SI-NEXT: v_or_b32_e32 v4, v5, v4
-; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v35
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v4, vcc, 0x3000000, v4
+; SI-NEXT: v_and_b32_e32 v43, 0xffff0000, v4
+; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v4
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v5, vcc, 3, v5
; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 8, v6
-; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v5, v9, v5
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v5, vcc, s6, v5
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_add_i32_e32 v4, vcc, 0x3000000, v4
-; SI-NEXT: v_and_b32_e32 v63, 0xffff0000, v4
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_add_i32_e32 v7, vcc, 3, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; SI-NEXT: v_or_b32_e32 v7, v55, v7
-; SI-NEXT: v_lshlrev_b32_e32 v61, 16, v4
-; SI-NEXT: v_and_b32_e32 v20, 0xffff0000, v9
-; SI-NEXT: v_and_b32_e32 v34, 0xffff0000, v10
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v10
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v11
-; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_lshlrev_b32_e32 v8, 8, v8
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_or_b32_e32 v11, v48, v11
-; SI-NEXT: v_lshlrev_b32_e32 v12, 8, v12
-; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v13
-; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_add_i32_e32 v6, vcc, 3, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; SI-NEXT: v_or_b32_e32 v6, v42, v6
-; SI-NEXT: v_or_b32_e32 v5, v6, v5
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v9
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_or_b32_e32 v9, v55, v9
+; SI-NEXT: v_or_b32_e32 v5, v9, v5
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v5, vcc, 0x3000000, v5
-; SI-NEXT: v_and_b32_e32 v60, 0xffff0000, v5
-; SI-NEXT: v_lshlrev_b32_e32 v58, 16, v5
-; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:232 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v6, vcc, 3, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v6
-; SI-NEXT: v_or_b32_e32 v6, v44, v6
-; SI-NEXT: v_add_i32_e32 v6, vcc, s6, v6
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_or_b32_e32 v6, v7, v6
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v6, vcc, 0x3000000, v6
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v14, 8, v14
-; SI-NEXT: v_and_b32_e32 v47, 0xffff0000, v6
-; SI-NEXT: v_lshlrev_b32_e32 v46, 16, v6
+; SI-NEXT: v_and_b32_e32 v40, 0xffff0000, v5
+; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v5
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v7, vcc, 3, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xff, v7
-; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v7, vcc, s6, v7
-; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
+; SI-NEXT: v_add_i32_e32 v9, vcc, 3, v9
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_or_b32_e32 v9, v42, v9
+; SI-NEXT: v_add_i32_e32 v9, vcc, s6, v9
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
+; SI-NEXT: v_or_b32_e32 v9, v10, v9
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v9, vcc, 0x3000000, v9
+; SI-NEXT: v_and_b32_e32 v51, 0xffff0000, v9
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v9
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_or_b32_e32 v8, v50, v8
-; SI-NEXT: v_or_b32_e32 v7, v8, v7
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v7, vcc, 0x3000000, v7
-; SI-NEXT: v_and_b32_e32 v45, 0xffff0000, v7
-; SI-NEXT: v_lshlrev_b32_e32 v41, 16, v7
+; SI-NEXT: v_add_i32_e32 v10, vcc, 3, v10
+; SI-NEXT: v_and_b32_e32 v10, 0xff, v10
+; SI-NEXT: v_or_b32_e32 v10, v11, v10
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v10, vcc, s6, v10
+; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v8, vcc, 3, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v8, v52, v8
-; SI-NEXT: v_add_i32_e32 v8, vcc, s6, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_or_b32_e32 v8, v11, v8
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v8, vcc, 0x3000000, v8
-; SI-NEXT: v_and_b32_e32 v40, 0xffff0000, v8
-; SI-NEXT: v_lshlrev_b32_e32 v54, 16, v8
+; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v11
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
+; SI-NEXT: v_or_b32_e32 v11, v48, v11
+; SI-NEXT: v_or_b32_e32 v10, v11, v10
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v10, vcc, 0x3000000, v10
+; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v10
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v10
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v11, vcc, 3, v11
; SI-NEXT: v_and_b32_e32 v11, 0xff, v11
-; SI-NEXT: v_or_b32_e32 v11, v12, v11
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:236 ; 4-byte Folded Reload
+; SI-NEXT: v_or_b32_e32 v11, v50, v11
; SI-NEXT: v_add_i32_e32 v11, vcc, s6, v11
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v12, vcc, 3, v12
-; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
-; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
-; SI-NEXT: v_or_b32_e32 v12, v36, v12
; SI-NEXT: v_or_b32_e32 v11, v12, v11
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v11, vcc, 0x3000000, v11
-; SI-NEXT: v_and_b32_e32 v53, 0xffff0000, v11
-; SI-NEXT: v_lshlrev_b32_e32 v51, 16, v11
-; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v2
-; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_and_b32_e32 v21, 0xffff0000, v11
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
+; SI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:316 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v0
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v12, vcc, 3, v12
; SI-NEXT: v_and_b32_e32 v12, 0xff, v12
-; SI-NEXT: v_or_b32_e32 v12, v38, v12
+; SI-NEXT: v_or_b32_e32 v12, v13, v12
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; SI-NEXT: v_add_i32_e32 v12, vcc, s6, v12
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_or_b32_e32 v12, v13, v12
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v12
-; SI-NEXT: v_and_b32_e32 v49, 0xffff0000, v12
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v12
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v13
; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
-; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v13, vcc, s6, v13
-; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
-; SI-NEXT: v_or_b32_e32 v14, v15, v14
-; SI-NEXT: v_or_b32_e32 v13, v14, v13
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v13
-; SI-NEXT: v_and_b32_e32 v27, 0xffff0000, v13
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v13
-; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v0
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_or_b32_e32 v13, v15, v13
+; SI-NEXT: v_or_b32_e32 v12, v13, v12
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:240 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
+; SI-NEXT: v_add_i32_e32 v12, vcc, 0x3000000, v12
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_add_i32_e32 v14, vcc, 3, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xff, v14
+; SI-NEXT: v_add_i32_e32 v13, vcc, 3, v13
+; SI-NEXT: v_and_b32_e32 v13, 0xff, v13
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_or_b32_e32 v14, v15, v14
+; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v14, vcc, s6, v14
-; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_add_i32_e32 v13, vcc, s6, v13
+; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v15
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v16, v15
-; SI-NEXT: v_or_b32_e32 v14, v15, v14
+; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:228 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
-; SI-NEXT: v_add_i32_e32 v14, vcc, 0x3000000, v14
+; SI-NEXT: v_add_i32_e32 v13, vcc, 0x3000000, v13
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_add_i32_e32 v15, vcc, 3, v15
; SI-NEXT: v_and_b32_e32 v15, 0xff, v15
@@ -92059,70 +92057,82 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: v_add_i32_e32 v15, vcc, 0x3000000, v15
; SI-NEXT: v_and_b32_e32 v16, 0xffff0000, v15
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v14
-; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:304 ; 4-byte Folded Spill
-; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
-; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_and_b32_e32 v15, 0xffff0000, v13
+; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v3
-; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v9
-; SI-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:320 ; 4-byte Folded Spill
+; SI-NEXT: v_and_b32_e32 v13, 0xffff0000, v12
+; SI-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:296 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_and_b32_e32 v13, 0xffff0000, v0
+; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v7
+; SI-NEXT: v_and_b32_e32 v16, 0xffff0000, v14
+; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:300 ; 4-byte Folded Spill
; SI-NEXT: .LBB110_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
-; SI-NEXT: v_mul_f32_e32 v3, 1.0, v23
-; SI-NEXT: v_mul_f32_e32 v4, 1.0, v29
-; SI-NEXT: v_mul_f32_e32 v5, 1.0, v51
-; SI-NEXT: v_mul_f32_e32 v6, 1.0, v54
-; SI-NEXT: v_mul_f32_e32 v7, 1.0, v41
-; SI-NEXT: v_mul_f32_e32 v8, 1.0, v46
-; SI-NEXT: v_mul_f32_e32 v9, 1.0, v58
-; SI-NEXT: v_mul_f32_e32 v10, 1.0, v61
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(9)
+; SI-NEXT: v_mul_f32_e32 v5, 1.0, v23
+; SI-NEXT: v_mul_f32_e32 v6, 1.0, v29
+; SI-NEXT: v_mul_f32_e32 v7, 1.0, v54
+; SI-NEXT: v_mul_f32_e32 v8, 1.0, v41
+; SI-NEXT: v_mul_f32_e32 v9, 1.0, v45
+; SI-NEXT: v_mul_f32_e32 v10, 1.0, v57
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mul_f32_e32 v12, 1.0, v62
; SI-NEXT: v_mul_f32_e32 v11, 1.0, v11
-; SI-NEXT: v_mul_f32_e32 v12, 1.0, v22
-; SI-NEXT: v_mul_f32_e32 v13, 1.0, v13
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mul_f32_e32 v14, 1.0, v15
-; SI-NEXT: v_mul_f32_e32 v15, 1.0, v16
-; SI-NEXT: v_mul_f32_e32 v16, 1.0, v33
-; SI-NEXT: s_waitcnt vmcnt(2)
+; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_mul_f32_e32 v14, 1.0, v33
+; SI-NEXT: v_mul_f32_e32 v15, 1.0, v15
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_mul_f32_e32 v0, 1.0, v0
; SI-NEXT: v_lshrrev_b32_e32 v0, 16, v0
-; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: s_waitcnt vmcnt(3)
+; SI-NEXT: v_mul_f32_e32 v2, 1.0, v2
+; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_mul_f32_e32 v1, 1.0, v1
; SI-NEXT: v_alignbit_b32 v0, v0, v1, 16
-; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
-; SI-NEXT: v_mul_f32_e32 v2, 1.0, v2
+; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
+; SI-NEXT: v_mul_f32_e32 v3, 1.0, v3
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_mul_f32_e32 v4, 1.0, v4
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_mul_f32_e32 v1, 1.0, v1
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v1
; SI-NEXT: v_alignbit_b32 v1, v1, v2, 16
-; SI-NEXT: v_mul_f32_e32 v2, 1.0, v27
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_mul_f32_e32 v2, 1.0, v2
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v2
; SI-NEXT: v_alignbit_b32 v2, v2, v3, 16
-; SI-NEXT: v_mul_f32_e32 v3, 1.0, v49
+; SI-NEXT: v_mul_f32_e32 v3, 1.0, v21
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v3
; SI-NEXT: v_alignbit_b32 v3, v3, v4, 16
-; SI-NEXT: v_mul_f32_e32 v4, 1.0, v53
+; SI-NEXT: v_mul_f32_e32 v4, 1.0, v27
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v4
; SI-NEXT: v_alignbit_b32 v4, v4, v5, 16
-; SI-NEXT: v_mul_f32_e32 v5, 1.0, v40
+; SI-NEXT: v_mul_f32_e32 v5, 1.0, v51
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v5
; SI-NEXT: v_alignbit_b32 v5, v5, v6, 16
-; SI-NEXT: v_mul_f32_e32 v6, 1.0, v45
+; SI-NEXT: v_mul_f32_e32 v6, 1.0, v40
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v6
; SI-NEXT: v_alignbit_b32 v6, v6, v7, 16
-; SI-NEXT: v_mul_f32_e32 v7, 1.0, v47
+; SI-NEXT: v_mul_f32_e32 v7, 1.0, v43
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v7
; SI-NEXT: v_alignbit_b32 v7, v7, v8, 16
-; SI-NEXT: v_mul_f32_e32 v8, 1.0, v60
+; SI-NEXT: v_mul_f32_e32 v8, 1.0, v56
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v8
; SI-NEXT: v_alignbit_b32 v8, v8, v9, 16
-; SI-NEXT: v_mul_f32_e32 v9, 1.0, v63
+; SI-NEXT: v_mul_f32_e32 v9, 1.0, v60
; SI-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v62, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v61, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
@@ -92141,23 +92151,25 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v9
; SI-NEXT: v_alignbit_b32 v9, v9, v10, 16
-; SI-NEXT: v_mul_f32_e32 v10, 1.0, v32
+; SI-NEXT: v_mul_f32_e32 v10, 1.0, v31
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v10
-; SI-NEXT: v_alignbit_b32 v10, v10, v11, 16
-; SI-NEXT: v_mul_f32_e32 v11, 1.0, v26
-; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v11
-; SI-NEXT: v_alignbit_b32 v11, v11, v12, 16
-; SI-NEXT: v_mul_f32_e32 v12, 1.0, v30
+; SI-NEXT: v_alignbit_b32 v10, v10, v12, 16
+; SI-NEXT: v_mul_f32_e32 v12, 1.0, v13
+; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_alignbit_b32 v11, v12, v11, 16
+; SI-NEXT: v_mul_f32_e32 v12, 1.0, v32
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_mul_f32_e32 v13, 1.0, v22
; SI-NEXT: v_alignbit_b32 v12, v12, v13, 16
-; SI-NEXT: v_mul_f32_e32 v13, 1.0, v24
+; SI-NEXT: v_mul_f32_e32 v13, 1.0, v30
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v13
; SI-NEXT: v_alignbit_b32 v13, v13, v14, 16
-; SI-NEXT: v_mul_f32_e32 v14, 1.0, v20
+; SI-NEXT: v_mul_f32_e32 v14, 1.0, v19
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v14
; SI-NEXT: v_alignbit_b32 v14, v14, v15, 16
-; SI-NEXT: v_mul_f32_e32 v15, 1.0, v34
+; SI-NEXT: v_mul_f32_e32 v15, 1.0, v16
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_mul_f32_e32 v16, 1.0, v24
; SI-NEXT: v_alignbit_b32 v15, v15, v16, 16
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -92183,16 +92195,16 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_store_dword v63, off, s[0:3], s32 offset:136 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:260 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:40
+; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:40
; VI-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:36
-; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:32
-; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:28
-; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:24
-; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:20
-; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:16
-; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:12
-; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:8
-; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:4
+; VI-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:32
+; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:28
+; VI-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:24
+; VI-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:20
+; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:16
+; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:12
+; VI-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:8
+; VI-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:4
; VI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; VI-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:240 ; 4-byte Folded Spill
@@ -92219,18 +92231,18 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: buffer_load_ushort v39, off, s[0:3], s32 offset:100
; VI-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:96
; VI-NEXT: buffer_load_ushort v49, off, s[0:3], s32 offset:92
-; VI-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:88
-; VI-NEXT: buffer_load_ushort v37, off, s[0:3], s32 offset:84
-; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:80
-; VI-NEXT: buffer_load_ushort v35, off, s[0:3], s32 offset:76
-; VI-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:72
-; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:68
-; VI-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:64
-; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:60
-; VI-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:56
-; VI-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:52
-; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:48
-; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:44
+; VI-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:88
+; VI-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:84
+; VI-NEXT: buffer_load_ushort v33, off, s[0:3], s32 offset:80
+; VI-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:76
+; VI-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:72
+; VI-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:68
+; VI-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:64
+; VI-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:60
+; VI-NEXT: buffer_load_ushort v41, off, s[0:3], s32 offset:56
+; VI-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:52
+; VI-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:48
+; VI-NEXT: buffer_load_ushort v42, off, s[0:3], s32 offset:44
; VI-NEXT: v_mov_b32_e32 v17, v14
; VI-NEXT: s_waitcnt vmcnt(14)
; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
@@ -92250,16 +92262,16 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; VI-NEXT: v_perm_b32 v8, v30, v55, s6
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; VI-NEXT: v_perm_b32 v9, v44, v42, s6
+; VI-NEXT: v_perm_b32 v9, v62, v59, s6
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
-; VI-NEXT: v_perm_b32 v10, v57, v46, s6
+; VI-NEXT: v_perm_b32 v10, v37, v35, s6
; VI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; VI-NEXT: s_waitcnt vmcnt(2)
-; VI-NEXT: v_perm_b32 v11, v58, v47, s6
+; VI-NEXT: v_perm_b32 v11, v42, v40, s6
; VI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
-; VI-NEXT: v_perm_b32 v12, v33, v62, s6
+; VI-NEXT: v_perm_b32 v12, v57, v46, s6
; VI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
-; VI-NEXT: v_perm_b32 v13, v35, v34, s6
+; VI-NEXT: v_perm_b32 v13, v34, v33, s6
; VI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
; VI-NEXT: v_perm_b32 v14, v49, v48, s6
; VI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
@@ -92275,16 +92287,16 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr27
; VI-NEXT: ; implicit-def: $vgpr30
; VI-NEXT: ; implicit-def: $vgpr55
-; VI-NEXT: ; implicit-def: $vgpr44
+; VI-NEXT: ; implicit-def: $vgpr62
+; VI-NEXT: ; implicit-def: $vgpr59
+; VI-NEXT: ; implicit-def: $vgpr37
+; VI-NEXT: ; implicit-def: $vgpr35
; VI-NEXT: ; implicit-def: $vgpr42
+; VI-NEXT: ; implicit-def: $vgpr40
; VI-NEXT: ; implicit-def: $vgpr57
; VI-NEXT: ; implicit-def: $vgpr46
-; VI-NEXT: ; implicit-def: $vgpr58
-; VI-NEXT: ; implicit-def: $vgpr47
-; VI-NEXT: ; implicit-def: $vgpr33
-; VI-NEXT: ; implicit-def: $vgpr62
-; VI-NEXT: ; implicit-def: $vgpr35
; VI-NEXT: ; implicit-def: $vgpr34
+; VI-NEXT: ; implicit-def: $vgpr33
; VI-NEXT: ; implicit-def: $vgpr49
; VI-NEXT: ; implicit-def: $vgpr48
; VI-NEXT: ; implicit-def: $vgpr51
@@ -92337,17 +92349,17 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v6, v6, v7
; VI-NEXT: v_perm_b32 v7, v28, v29, s6
; VI-NEXT: v_or_b32_e32 v7, v7, v8
-; VI-NEXT: v_perm_b32 v8, v41, v40, s6
+; VI-NEXT: v_perm_b32 v8, v45, v44, s6
; VI-NEXT: v_or_b32_e32 v8, v8, v9
-; VI-NEXT: v_perm_b32 v9, v45, v43, s6
+; VI-NEXT: v_perm_b32 v9, v63, v60, s6
; VI-NEXT: v_or_b32_e32 v9, v9, v10
-; VI-NEXT: v_perm_b32 v10, v61, v60, s6
+; VI-NEXT: v_perm_b32 v10, v61, v56, s6
; VI-NEXT: v_or_b32_e32 v10, v10, v11
-; VI-NEXT: v_perm_b32 v11, v59, v56, s6
+; VI-NEXT: v_perm_b32 v11, v43, v41, s6
; VI-NEXT: v_or_b32_e32 v11, v11, v12
-; VI-NEXT: v_perm_b32 v12, v31, v63, s6
+; VI-NEXT: v_perm_b32 v12, v58, v47, s6
; VI-NEXT: v_or_b32_e32 v12, v12, v13
-; VI-NEXT: v_perm_b32 v13, v37, v36, s6
+; VI-NEXT: v_perm_b32 v13, v36, v31, s6
; VI-NEXT: v_or_b32_e32 v13, v13, v14
; VI-NEXT: v_perm_b32 v14, v39, v38, s6
; VI-NEXT: v_perm_b32 v16, v32, v54, s6
@@ -92392,18 +92404,18 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: ; implicit-def: $vgpr25
; VI-NEXT: ; implicit-def: $vgpr28
; VI-NEXT: ; implicit-def: $vgpr29
-; VI-NEXT: ; implicit-def: $vgpr41
-; VI-NEXT: ; implicit-def: $vgpr40
; VI-NEXT: ; implicit-def: $vgpr45
-; VI-NEXT: ; implicit-def: $vgpr43
-; VI-NEXT: ; implicit-def: $vgpr61
+; VI-NEXT: ; implicit-def: $vgpr44
+; VI-NEXT: ; implicit-def: $vgpr63
; VI-NEXT: ; implicit-def: $vgpr60
-; VI-NEXT: ; implicit-def: $vgpr59
+; VI-NEXT: ; implicit-def: $vgpr61
; VI-NEXT: ; implicit-def: $vgpr56
-; VI-NEXT: ; implicit-def: $vgpr31
-; VI-NEXT: ; implicit-def: $vgpr63
-; VI-NEXT: ; implicit-def: $vgpr37
+; VI-NEXT: ; implicit-def: $vgpr43
+; VI-NEXT: ; implicit-def: $vgpr41
+; VI-NEXT: ; implicit-def: $vgpr58
+; VI-NEXT: ; implicit-def: $vgpr47
; VI-NEXT: ; implicit-def: $vgpr36
+; VI-NEXT: ; implicit-def: $vgpr31
; VI-NEXT: ; implicit-def: $vgpr39
; VI-NEXT: ; implicit-def: $vgpr38
; VI-NEXT: ; implicit-def: $vgpr53
@@ -92434,48 +92446,48 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_add_u16_sdwa v32, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_u16_sdwa v38, v1, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(10)
-; VI-NEXT: v_add_u16_e32 v0, 3, v37
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v36
-; VI-NEXT: v_or_b32_sdwa v36, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v36
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v31
+; VI-NEXT: v_or_b32_sdwa v31, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(8)
-; VI-NEXT: v_add_u16_e32 v0, 3, v35
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v34
+; VI-NEXT: v_add_u16_e32 v0, 3, v34
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v33
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v12, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(6)
-; VI-NEXT: v_add_u16_e32 v0, 3, v31
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v63
-; VI-NEXT: v_or_b32_sdwa v31, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v58
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v47
+; VI-NEXT: v_or_b32_sdwa v33, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(4)
-; VI-NEXT: v_add_u16_e32 v0, 3, v33
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v62
+; VI-NEXT: v_add_u16_e32 v0, 3, v57
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v46
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v11, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: s_waitcnt vmcnt(2)
-; VI-NEXT: v_add_u16_e32 v0, 3, v59
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v56
-; VI-NEXT: v_or_b32_sdwa v33, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v43
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v41
+; VI-NEXT: v_or_b32_sdwa v34, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: s_waitcnt vmcnt(0)
-; VI-NEXT: v_add_u16_e32 v0, 3, v58
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v47
+; VI-NEXT: v_add_u16_e32 v0, 3, v42
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v40
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v10, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_u16_e32 v0, 3, v61
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v60
-; VI-NEXT: v_or_b32_sdwa v34, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: v_add_u16_e32 v0, 3, v57
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v46
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v56
+; VI-NEXT: v_or_b32_sdwa v36, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; VI-NEXT: v_add_u16_e32 v0, 3, v37
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v35
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v9, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_u16_e32 v0, 3, v45
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v43
+; VI-NEXT: v_add_u16_e32 v0, 3, v63
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v60
; VI-NEXT: v_or_b32_sdwa v35, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; VI-NEXT: v_add_u16_e32 v0, 3, v44
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v42
+; VI-NEXT: v_add_u16_e32 v0, 3, v62
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v59
; VI-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_sdwa v8, v0, v3 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_u16_e32 v0, 3, v41
-; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v40
+; VI-NEXT: v_add_u16_e32 v0, 3, v45
+; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v44
; VI-NEXT: v_or_b32_sdwa v37, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; VI-NEXT: v_add_u16_e32 v0, 3, v30
; VI-NEXT: v_lshlrev_b16_e32 v1, 8, v55
@@ -92528,13 +92540,13 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; VI-NEXT: v_or_b32_e32 v8, v16, v8
; VI-NEXT: v_add_u16_e32 v16, 0x300, v35
; VI-NEXT: v_or_b32_e32 v9, v16, v9
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v34
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v36
; VI-NEXT: v_or_b32_e32 v10, v16, v10
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v33
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v34
; VI-NEXT: v_or_b32_e32 v11, v16, v11
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v31
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v33
; VI-NEXT: v_or_b32_e32 v12, v16, v12
-; VI-NEXT: v_add_u16_e32 v16, 0x300, v36
+; VI-NEXT: v_add_u16_e32 v16, 0x300, v31
; VI-NEXT: v_or_b32_e32 v13, v16, v13
; VI-NEXT: s_waitcnt vmcnt(2)
; VI-NEXT: v_lshlrev_b16_e32 v21, 8, v21
@@ -92642,16 +92654,16 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v14, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:256 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_ushort v48, off, s[0:3], s32 offset:36
-; GFX9-NEXT: buffer_load_ushort v60, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:24
-; GFX9-NEXT: buffer_load_ushort v63, off, s[0:3], s32 offset:20
-; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:36
+; GFX9-NEXT: buffer_load_ushort v62, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:20
+; GFX9-NEXT: buffer_load_ushort v59, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_load_ushort v61, off, s[0:3], s32 offset:4
; GFX9-NEXT: buffer_load_ushort v53, off, s[0:3], s32 offset:124
; GFX9-NEXT: buffer_load_ushort v54, off, s[0:3], s32 offset:120
; GFX9-NEXT: buffer_load_ushort v52, off, s[0:3], s32 offset:116
-; GFX9-NEXT: buffer_load_ushort v36, off, s[0:3], s32
+; GFX9-NEXT: buffer_load_ushort v33, off, s[0:3], s32
; GFX9-NEXT: v_mov_b32_e32 v19, v12
; GFX9-NEXT: buffer_store_dword v11, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; GFX9-NEXT: buffer_store_dword v10, off, s[0:3], s32 offset:228 ; 4-byte Folded Spill
@@ -92679,11 +92691,11 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_ushort v10, off, s[0:3], s32 offset:104
; GFX9-NEXT: buffer_load_ushort v11, off, s[0:3], s32 offset:112
; GFX9-NEXT: buffer_load_ushort v12, off, s[0:3], s32 offset:128
-; GFX9-NEXT: buffer_load_ushort v34, off, s[0:3], s32 offset:108
-; GFX9-NEXT: buffer_load_ushort v31, off, s[0:3], s32 offset:100
-; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:92
-; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:88
-; GFX9-NEXT: buffer_load_ushort v58, off, s[0:3], s32 offset:84
+; GFX9-NEXT: buffer_load_ushort v56, off, s[0:3], s32 offset:108
+; GFX9-NEXT: buffer_load_ushort v57, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_load_ushort v45, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_load_ushort v46, off, s[0:3], s32 offset:88
+; GFX9-NEXT: buffer_load_ushort v47, off, s[0:3], s32 offset:84
; GFX9-NEXT: buffer_load_ushort v43, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v44, off, s[0:3], s32 offset:68
; GFX9-NEXT: buffer_load_ushort v40, off, s[0:3], s32 offset:60
@@ -92693,15 +92705,15 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_waitcnt vmcnt(23)
; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; GFX9-NEXT: s_waitcnt vmcnt(22)
-; GFX9-NEXT: v_lshlrev_b16_e32 v56, 8, v1
+; GFX9-NEXT: v_lshlrev_b16_e32 v60, 8, v1
; GFX9-NEXT: s_waitcnt vmcnt(21)
-; GFX9-NEXT: v_lshlrev_b16_e32 v45, 8, v2
+; GFX9-NEXT: v_lshlrev_b16_e32 v58, 8, v2
; GFX9-NEXT: s_waitcnt vmcnt(20)
-; GFX9-NEXT: v_lshlrev_b16_e32 v61, 8, v3
+; GFX9-NEXT: v_lshlrev_b16_e32 v63, 8, v3
; GFX9-NEXT: s_waitcnt vmcnt(19)
; GFX9-NEXT: v_lshlrev_b16_e32 v38, 8, v4
; GFX9-NEXT: s_waitcnt vmcnt(18)
-; GFX9-NEXT: v_lshlrev_b16_e32 v33, 8, v5
+; GFX9-NEXT: v_lshlrev_b16_e32 v48, 8, v5
; GFX9-NEXT: s_waitcnt vmcnt(17)
; GFX9-NEXT: v_lshlrev_b16_e32 v37, 8, v6
; GFX9-NEXT: s_waitcnt vmcnt(16)
@@ -92727,33 +92739,33 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: s_mov_b32 s7, 0x5040100
; GFX9-NEXT: v_perm_b32 v6, v22, v23, s6
; GFX9-NEXT: v_perm_b32 v7, v26, v27, s6
-; GFX9-NEXT: v_perm_b32 v8, v30, v36, s6
-; GFX9-NEXT: v_or_b32_sdwa v9, v47, v45 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
-; GFX9-NEXT: v_perm_b32 v10, v63, v62, s6
+; GFX9-NEXT: v_perm_b32 v8, v30, v33, s6
+; GFX9-NEXT: v_or_b32_sdwa v9, v59, v58 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v10, v34, v36, s6
; GFX9-NEXT: s_waitcnt vmcnt(2)
-; GFX9-NEXT: v_or_b32_sdwa v11, v55, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v11, v55, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v12, v42, v41, s6
; GFX9-NEXT: v_or_b32_sdwa v13, v43, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
-; GFX9-NEXT: v_perm_b32 v14, v58, v57, s6
-; GFX9-NEXT: v_or_b32_sdwa v15, v34, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_perm_b32 v14, v47, v46, s6
+; GFX9-NEXT: v_or_b32_sdwa v15, v56, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: ; implicit-def: $vgpr22
; GFX9-NEXT: ; implicit-def: $vgpr23
; GFX9-NEXT: ; implicit-def: $vgpr26
; GFX9-NEXT: ; implicit-def: $vgpr27
; GFX9-NEXT: ; implicit-def: $vgpr30
+; GFX9-NEXT: ; implicit-def: $vgpr33
+; GFX9-NEXT: ; implicit-def: $vgpr59
+; GFX9-NEXT: ; implicit-def: $vgpr34
; GFX9-NEXT: ; implicit-def: $vgpr36
-; GFX9-NEXT: ; implicit-def: $vgpr47
-; GFX9-NEXT: ; implicit-def: $vgpr63
-; GFX9-NEXT: ; implicit-def: $vgpr62
; GFX9-NEXT: ; implicit-def: $vgpr55
; GFX9-NEXT: ; implicit-def: $vgpr42
; GFX9-NEXT: ; implicit-def: $vgpr41
; GFX9-NEXT: ; implicit-def: $vgpr43
+; GFX9-NEXT: ; implicit-def: $vgpr47
+; GFX9-NEXT: ; implicit-def: $vgpr46
+; GFX9-NEXT: ; implicit-def: $vgpr56
; GFX9-NEXT: ; implicit-def: $vgpr58
-; GFX9-NEXT: ; implicit-def: $vgpr57
-; GFX9-NEXT: ; implicit-def: $vgpr34
-; GFX9-NEXT: ; implicit-def: $vgpr45
-; GFX9-NEXT: ; implicit-def: $vgpr33
+; GFX9-NEXT: ; implicit-def: $vgpr48
; GFX9-NEXT: ; implicit-def: $vgpr35
; GFX9-NEXT: ; implicit-def: $vgpr39
; GFX9-NEXT: s_waitcnt vmcnt(0)
@@ -92805,19 +92817,19 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v6, v7, v6, s7
; GFX9-NEXT: v_perm_b32 v7, v28, v29, s6
; GFX9-NEXT: v_perm_b32 v7, v8, v7, s7
-; GFX9-NEXT: v_or_b32_sdwa v8, v59, v56 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v8, v61, v60 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v8, v9, v8, s7
-; GFX9-NEXT: v_or_b32_sdwa v9, v60, v61 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v9, v62, v63 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v9, v9, v10, s7
-; GFX9-NEXT: v_or_b32_sdwa v10, v48, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v10, v31, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v10, v11, v10, s7
; GFX9-NEXT: v_or_b32_sdwa v11, v40, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v11, v11, v12, s7
; GFX9-NEXT: v_or_b32_sdwa v12, v44, v51 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v12, v13, v12, s7
-; GFX9-NEXT: v_or_b32_sdwa v13, v46, v50 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v13, v45, v50 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v13, v13, v14, s7
-; GFX9-NEXT: v_or_b32_sdwa v14, v31, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
+; GFX9-NEXT: v_or_b32_sdwa v14, v57, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v14, v15, v14, s7
; GFX9-NEXT: v_or_b32_sdwa v15, v53, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NEXT: v_perm_b32 v15, v15, v16, s7
@@ -92861,16 +92873,16 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: ; implicit-def: $vgpr25
; GFX9-NEXT: ; implicit-def: $vgpr28
; GFX9-NEXT: ; implicit-def: $vgpr29
-; GFX9-NEXT: ; implicit-def: $vgpr59
-; GFX9-NEXT: ; implicit-def: $vgpr60
-; GFX9-NEXT: ; implicit-def: $vgpr48
+; GFX9-NEXT: ; implicit-def: $vgpr61
+; GFX9-NEXT: ; implicit-def: $vgpr62
+; GFX9-NEXT: ; implicit-def: $vgpr31
; GFX9-NEXT: ; implicit-def: $vgpr40
; GFX9-NEXT: ; implicit-def: $vgpr44
-; GFX9-NEXT: ; implicit-def: $vgpr46
-; GFX9-NEXT: ; implicit-def: $vgpr31
+; GFX9-NEXT: ; implicit-def: $vgpr45
+; GFX9-NEXT: ; implicit-def: $vgpr57
; GFX9-NEXT: ; implicit-def: $vgpr53
-; GFX9-NEXT: ; implicit-def: $vgpr56
-; GFX9-NEXT: ; implicit-def: $vgpr61
+; GFX9-NEXT: ; implicit-def: $vgpr60
+; GFX9-NEXT: ; implicit-def: $vgpr63
; GFX9-NEXT: ; implicit-def: $vgpr38
; GFX9-NEXT: ; implicit-def: $vgpr37
; GFX9-NEXT: ; implicit-def: $vgpr51
@@ -92886,19 +92898,18 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v1, 3, v53
; GFX9-NEXT: v_or_b32_sdwa v1, v32, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: s_waitcnt vmcnt(9)
-; GFX9-NEXT: v_add_u16_e32 v2, 3, v31
; GFX9-NEXT: v_add_u16_e32 v15, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v31, 0x300, v1
+; GFX9-NEXT: v_add_u16_e32 v32, 0x300, v1
; GFX9-NEXT: s_waitcnt vmcnt(6)
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v58
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v57
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v47
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v46
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v13, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v46
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v45
+; GFX9-NEXT: v_add_u16_e32 v2, 3, v57
; GFX9-NEXT: v_or_b32_sdwa v0, v50, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v3, 3, v34
-; GFX9-NEXT: v_add_u16_e32 v34, 0x300, v0
+; GFX9-NEXT: v_or_b32_sdwa v2, v49, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v49, 0x300, v0
; GFX9-NEXT: s_waitcnt vmcnt(4)
; GFX9-NEXT: v_add_u16_e32 v0, 3, v44
; GFX9-NEXT: v_or_b32_sdwa v0, v51, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
@@ -92914,33 +92925,32 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: v_add_u16_e32 v0, 3, v40
; GFX9-NEXT: v_or_b32_sdwa v0, v37, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v37, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v48
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v31
; GFX9-NEXT: v_or_b32_sdwa v0, v38, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v10, 0x300, v0
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_add_u16_e32 v0, 3, v55
-; GFX9-NEXT: v_or_b32_sdwa v0, v33, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v33, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v63
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v62
+; GFX9-NEXT: v_or_b32_sdwa v0, v48, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v31, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v34
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v36
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v9, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v60
-; GFX9-NEXT: v_or_b32_sdwa v0, v61, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v38, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v59
-; GFX9-NEXT: v_or_b32_sdwa v0, v56, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v62
+; GFX9-NEXT: v_or_b32_sdwa v0, v63, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v34, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v61
+; GFX9-NEXT: v_or_b32_sdwa v0, v60, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v8, 0x300, v0
-; GFX9-NEXT: v_add_u16_e32 v0, 3, v47
-; GFX9-NEXT: v_or_b32_sdwa v0, v45, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_or_b32_sdwa v3, v39, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX9-NEXT: v_add_u16_e32 v39, 0x300, v0
+; GFX9-NEXT: v_add_u16_e32 v0, 3, v59
+; GFX9-NEXT: v_or_b32_sdwa v0, v58, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v36, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v28
; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v29
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v7, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v30
-; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v36
+; GFX9-NEXT: v_lshlrev_b16_e32 v1, 8, v33
; GFX9-NEXT: v_or_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NEXT: v_add_u16_e32 v28, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v24
@@ -92965,21 +92975,22 @@ define <32 x bfloat> @bitcast_v64i8_to_v32bf16(<64 x i8> %a, i32 %b) #0 {
; GFX9-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-NEXT: v_add_u16_e32 v4, 0x300, v0
; GFX9-NEXT: v_add_u16_e32 v0, 3, v18
-; GFX9-NEXT: v_add_u16_e32 v32, 0x300, v3
-; GFX9-NEXT: v_or_b32_sdwa v2, v49, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v3, 3, v56
+; GFX9-NEXT: v_or_b32_sdwa v3, v39, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
+; GFX9-NEXT: v_add_u16_e32 v39, 0x300, v3
; GFX9-NEXT: v_add_u16_e32 v14, 0x300, v2
; GFX9-NEXT: s_mov_b32 s6, 0x5040100
; GFX9-NEXT: v_perm_b32 v5, v20, v5, s6
; GFX9-NEXT: v_perm_b32 v6, v24, v6, s6
; GFX9-NEXT: v_perm_b32 v7, v28, v7, s6
-; GFX9-NEXT: v_perm_b32 v8, v39, v8, s6
-; GFX9-NEXT: v_perm_b32 v9, v38, v9, s6
-; GFX9-NEXT: v_perm_b32 v10, v33, v10, s6
+; GFX9-NEXT: v_perm_b32 v8, v36, v8, s6
+; GFX9-NEXT: v_perm_b32 v9, v34, v9, s6
+; GFX9-NEXT: v_perm_b32 v10, v31, v10, s6
; GFX9-NEXT: v_perm_b32 v11, v37, v11, s6
; GFX9-NEXT: v_perm_b32 v12, v35, v12, s6
-; GFX9-NEXT: v_perm_b32 v13, v34, v13, s6
-; GFX9-NEXT: v_perm_b32 v14, v32, v14, s6
-; GFX9-NEXT: v_perm_b32 v15, v31, v15, s6
+; GFX9-NEXT: v_perm_b32 v13, v49, v13, s6
+; GFX9-NEXT: v_perm_b32 v14, v39, v14, s6
+; GFX9-NEXT: v_perm_b32 v15, v32, v15, s6
; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; GFX9-NEXT: s_waitcnt vmcnt(2)
@@ -94545,10 +94556,11 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; VI-NEXT: v_readfirstlane_b32 s7, v21
; VI-NEXT: v_readfirstlane_b32 s9, v20
; VI-NEXT: v_readfirstlane_b32 s6, v19
-; VI-NEXT: v_readfirstlane_b32 s44, v18
-; VI-NEXT: v_readfirstlane_b32 s42, v17
-; VI-NEXT: v_readfirstlane_b32 s43, v16
-; VI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; VI-NEXT: v_readfirstlane_b32 s56, v18
+; VI-NEXT: v_readfirstlane_b32 s43, v17
+; VI-NEXT: v_readfirstlane_b32 s46, v16
+; VI-NEXT: v_readfirstlane_b32 s45, v12
+; VI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
; VI-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:8
; VI-NEXT: buffer_load_ushort v18, off, s[0:3], s32 offset:4
; VI-NEXT: buffer_load_ushort v16, off, s[0:3], s32
@@ -94568,24 +94580,23 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; VI-NEXT: buffer_load_ushort v25, off, s[0:3], s32 offset:20
; VI-NEXT: buffer_load_ushort v19, off, s[0:3], s32 offset:16
; VI-NEXT: buffer_load_ushort v22, off, s[0:3], s32 offset:12
-; VI-NEXT: v_readfirstlane_b32 s74, v15
-; VI-NEXT: v_readfirstlane_b32 s76, v14
-; VI-NEXT: v_readfirstlane_b32 s72, v13
-; VI-NEXT: v_readfirstlane_b32 s75, v12
-; VI-NEXT: v_readfirstlane_b32 s62, v11
-; VI-NEXT: v_readfirstlane_b32 s73, v10
-; VI-NEXT: v_readfirstlane_b32 s60, v9
-; VI-NEXT: v_readfirstlane_b32 s63, v8
-; VI-NEXT: v_readfirstlane_b32 s58, v7
-; VI-NEXT: v_readfirstlane_b32 s61, v6
-; VI-NEXT: v_readfirstlane_b32 s47, v5
-; VI-NEXT: v_readfirstlane_b32 s59, v4
-; VI-NEXT: v_readfirstlane_b32 s46, v3
-; VI-NEXT: v_readfirstlane_b32 s56, v2
-; VI-NEXT: v_readfirstlane_b32 s45, v1
-; VI-NEXT: v_readfirstlane_b32 s57, v0
+; VI-NEXT: v_readfirstlane_b32 s44, v15
+; VI-NEXT: v_readfirstlane_b32 s47, v14
+; VI-NEXT: v_readfirstlane_b32 s42, v13
+; VI-NEXT: v_readfirstlane_b32 s74, v11
+; VI-NEXT: v_readfirstlane_b32 s76, v10
+; VI-NEXT: v_readfirstlane_b32 s72, v9
+; VI-NEXT: v_readfirstlane_b32 s75, v8
+; VI-NEXT: v_readfirstlane_b32 s62, v7
+; VI-NEXT: v_readfirstlane_b32 s73, v6
+; VI-NEXT: v_readfirstlane_b32 s59, v5
+; VI-NEXT: v_readfirstlane_b32 s63, v4
+; VI-NEXT: v_readfirstlane_b32 s58, v3
+; VI-NEXT: v_readfirstlane_b32 s60, v2
+; VI-NEXT: v_readfirstlane_b32 s57, v1
+; VI-NEXT: v_readfirstlane_b32 s61, v0
; VI-NEXT: s_waitcnt vmcnt(14)
-; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; VI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; VI-NEXT: s_and_b64 s[4:5], vcc, exec
; VI-NEXT: s_cbranch_scc0 .LBB111_2
; VI-NEXT: ; %bb.1: ; %cmp.false
@@ -94607,41 +94618,41 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; VI-NEXT: v_perm_b32 v3, s26, v3, v11
; VI-NEXT: v_perm_b32 v2, s24, v2, v11
; VI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; VI-NEXT: v_mov_b32_e32 v4, s45
+; VI-NEXT: v_mov_b32_e32 v4, s57
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: v_mov_b32_e32 v3, s29
-; VI-NEXT: v_perm_b32 v4, s57, v4, v11
+; VI-NEXT: v_perm_b32 v4, s61, v4, v11
; VI-NEXT: v_perm_b32 v3, s28, v3, v11
; VI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; VI-NEXT: v_mov_b32_e32 v5, s47
+; VI-NEXT: v_mov_b32_e32 v5, s59
; VI-NEXT: v_or_b32_e32 v3, v3, v4
-; VI-NEXT: v_mov_b32_e32 v4, s46
-; VI-NEXT: v_perm_b32 v5, s59, v5, v11
-; VI-NEXT: v_perm_b32 v4, s56, v4, v11
+; VI-NEXT: v_mov_b32_e32 v4, s58
+; VI-NEXT: v_perm_b32 v5, s63, v5, v11
+; VI-NEXT: v_perm_b32 v4, s60, v4, v11
; VI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
-; VI-NEXT: v_mov_b32_e32 v6, s60
+; VI-NEXT: v_mov_b32_e32 v6, s72
; VI-NEXT: v_or_b32_e32 v4, v4, v5
-; VI-NEXT: v_mov_b32_e32 v5, s58
-; VI-NEXT: v_perm_b32 v6, s63, v6, v11
-; VI-NEXT: v_perm_b32 v5, s61, v5, v11
+; VI-NEXT: v_mov_b32_e32 v5, s62
+; VI-NEXT: v_perm_b32 v6, s75, v6, v11
+; VI-NEXT: v_perm_b32 v5, s73, v5, v11
; VI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
-; VI-NEXT: v_mov_b32_e32 v7, s72
+; VI-NEXT: v_mov_b32_e32 v7, s42
; VI-NEXT: v_or_b32_e32 v5, v5, v6
-; VI-NEXT: v_mov_b32_e32 v6, s62
-; VI-NEXT: v_perm_b32 v7, s75, v7, v11
-; VI-NEXT: v_perm_b32 v6, s73, v6, v11
+; VI-NEXT: v_mov_b32_e32 v6, s74
+; VI-NEXT: v_perm_b32 v7, s45, v7, v11
+; VI-NEXT: v_perm_b32 v6, s76, v6, v11
; VI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
-; VI-NEXT: v_mov_b32_e32 v8, s42
+; VI-NEXT: v_mov_b32_e32 v8, s43
; VI-NEXT: v_or_b32_e32 v6, v6, v7
-; VI-NEXT: v_mov_b32_e32 v7, s74
-; VI-NEXT: v_perm_b32 v8, s43, v8, v11
-; VI-NEXT: v_perm_b32 v7, s76, v7, v11
+; VI-NEXT: v_mov_b32_e32 v7, s44
+; VI-NEXT: v_perm_b32 v8, s46, v8, v11
+; VI-NEXT: v_perm_b32 v7, s47, v7, v11
; VI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; VI-NEXT: v_mov_b32_e32 v9, s7
; VI-NEXT: v_or_b32_e32 v7, v7, v8
; VI-NEXT: v_mov_b32_e32 v8, s6
; VI-NEXT: v_perm_b32 v9, s9, v9, v11
-; VI-NEXT: v_perm_b32 v8, s44, v8, v11
+; VI-NEXT: v_perm_b32 v8, s56, v8, v11
; VI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; VI-NEXT: v_mov_b32_e32 v10, s10
; VI-NEXT: v_or_b32_e32 v8, v8, v9
@@ -94732,36 +94743,36 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; VI-NEXT: s_add_i32 s9, s9, 3
; VI-NEXT: v_mov_b32_e32 v1, s7
; VI-NEXT: v_perm_b32 v22, s9, v1, v0
-; VI-NEXT: s_add_i32 s44, s44, 3
+; VI-NEXT: s_add_i32 s56, s56, 3
; VI-NEXT: v_mov_b32_e32 v1, s6
-; VI-NEXT: v_perm_b32 v23, s44, v1, v0
-; VI-NEXT: s_add_i32 s43, s43, 3
+; VI-NEXT: v_perm_b32 v23, s56, v1, v0
+; VI-NEXT: s_add_i32 s46, s46, 3
+; VI-NEXT: v_mov_b32_e32 v1, s43
+; VI-NEXT: v_perm_b32 v7, s46, v1, v0
+; VI-NEXT: s_add_i32 s47, s47, 3
+; VI-NEXT: v_mov_b32_e32 v1, s44
+; VI-NEXT: v_perm_b32 v24, s47, v1, v0
+; VI-NEXT: s_add_i32 s45, s45, 3
; VI-NEXT: v_mov_b32_e32 v1, s42
-; VI-NEXT: v_perm_b32 v7, s43, v1, v0
+; VI-NEXT: v_perm_b32 v6, s45, v1, v0
; VI-NEXT: s_add_i32 s76, s76, 3
; VI-NEXT: v_mov_b32_e32 v1, s74
-; VI-NEXT: v_perm_b32 v24, s76, v1, v0
+; VI-NEXT: v_perm_b32 v25, s76, v1, v0
; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: v_mov_b32_e32 v1, s72
-; VI-NEXT: v_perm_b32 v6, s75, v1, v0
+; VI-NEXT: v_perm_b32 v5, s75, v1, v0
; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: v_mov_b32_e32 v1, s62
-; VI-NEXT: v_perm_b32 v25, s73, v1, v0
+; VI-NEXT: v_perm_b32 v26, s73, v1, v0
; VI-NEXT: s_add_i32 s63, s63, 3
-; VI-NEXT: v_mov_b32_e32 v1, s60
-; VI-NEXT: v_perm_b32 v5, s63, v1, v0
-; VI-NEXT: s_add_i32 s61, s61, 3
+; VI-NEXT: v_mov_b32_e32 v1, s59
+; VI-NEXT: v_perm_b32 v4, s63, v1, v0
+; VI-NEXT: s_add_i32 s60, s60, 3
; VI-NEXT: v_mov_b32_e32 v1, s58
-; VI-NEXT: v_perm_b32 v26, s61, v1, v0
-; VI-NEXT: s_add_i32 s59, s59, 3
-; VI-NEXT: v_mov_b32_e32 v1, s47
-; VI-NEXT: v_perm_b32 v4, s59, v1, v0
-; VI-NEXT: s_add_i32 s56, s56, 3
-; VI-NEXT: v_mov_b32_e32 v1, s46
-; VI-NEXT: v_perm_b32 v27, s56, v1, v0
-; VI-NEXT: s_add_i32 s57, s57, 3
-; VI-NEXT: v_mov_b32_e32 v1, s45
-; VI-NEXT: v_perm_b32 v3, s57, v1, v0
+; VI-NEXT: v_perm_b32 v27, s60, v1, v0
+; VI-NEXT: s_add_i32 s61, s61, 3
+; VI-NEXT: v_mov_b32_e32 v1, s57
+; VI-NEXT: v_perm_b32 v3, s61, v1, v0
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: v_mov_b32_e32 v1, s29
; VI-NEXT: v_perm_b32 v28, s28, v1, v0
@@ -94857,9 +94868,9 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9: ; %bb.0:
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX9-NEXT: v_readfirstlane_b32 s72, v30
-; GFX9-NEXT: v_readfirstlane_b32 s47, v29
+; GFX9-NEXT: v_readfirstlane_b32 s56, v29
; GFX9-NEXT: v_readfirstlane_b32 s57, v28
-; GFX9-NEXT: v_readfirstlane_b32 s60, v27
+; GFX9-NEXT: v_readfirstlane_b32 s61, v27
; GFX9-NEXT: v_readfirstlane_b32 s62, v26
; GFX9-NEXT: v_readfirstlane_b32 s7, v25
; GFX9-NEXT: v_readfirstlane_b32 s46, v24
@@ -94871,7 +94882,8 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: v_readfirstlane_b32 s10, v18
; GFX9-NEXT: v_readfirstlane_b32 s11, v17
; GFX9-NEXT: v_readfirstlane_b32 s12, v16
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; GFX9-NEXT: v_readfirstlane_b32 s15, v12
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:76
; GFX9-NEXT: buffer_load_ushort v16, off, s[0:3], s32 offset:8
; GFX9-NEXT: buffer_load_ushort v17, off, s[0:3], s32 offset:4
; GFX9-NEXT: buffer_load_ushort v18, off, s[0:3], s32
@@ -94892,23 +94904,22 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: buffer_load_ushort v26, off, s[0:3], s32 offset:16
; GFX9-NEXT: buffer_load_ushort v29, off, s[0:3], s32 offset:12
; GFX9-NEXT: v_readfirstlane_b32 s13, v15
-; GFX9-NEXT: v_readfirstlane_b32 s76, v14
+; GFX9-NEXT: v_readfirstlane_b32 s73, v14
; GFX9-NEXT: v_readfirstlane_b32 s14, v13
-; GFX9-NEXT: v_readfirstlane_b32 s15, v12
; GFX9-NEXT: v_readfirstlane_b32 s40, v11
-; GFX9-NEXT: v_readfirstlane_b32 s75, v10
+; GFX9-NEXT: v_readfirstlane_b32 s76, v10
; GFX9-NEXT: v_readfirstlane_b32 s41, v9
; GFX9-NEXT: v_readfirstlane_b32 s42, v8
; GFX9-NEXT: v_readfirstlane_b32 s43, v7
-; GFX9-NEXT: v_readfirstlane_b32 s74, v6
+; GFX9-NEXT: v_readfirstlane_b32 s75, v6
; GFX9-NEXT: v_readfirstlane_b32 s44, v5
; GFX9-NEXT: v_readfirstlane_b32 s45, v4
-; GFX9-NEXT: v_readfirstlane_b32 s56, v3
-; GFX9-NEXT: v_readfirstlane_b32 s73, v2
-; GFX9-NEXT: v_readfirstlane_b32 s61, v1
+; GFX9-NEXT: v_readfirstlane_b32 s47, v3
+; GFX9-NEXT: v_readfirstlane_b32 s74, v2
+; GFX9-NEXT: v_readfirstlane_b32 s60, v1
; GFX9-NEXT: v_readfirstlane_b32 s63, v0
; GFX9-NEXT: s_waitcnt vmcnt(19)
-; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v35
+; GFX9-NEXT: v_cmp_ne_u32_e32 vcc, 0, v12
; GFX9-NEXT: s_and_b64 s[4:5], vcc, exec
; GFX9-NEXT: s_cbranch_scc0 .LBB111_2
; GFX9-NEXT: ; %bb.1: ; %cmp.false
@@ -94930,7 +94941,7 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: v_perm_b32 v3, s26, v3, v11
; GFX9-NEXT: v_perm_b32 v2, s24, v2, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; GFX9-NEXT: v_mov_b32_e32 v4, s61
+; GFX9-NEXT: v_mov_b32_e32 v4, s60
; GFX9-NEXT: v_or_b32_sdwa v2, v3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: v_perm_b32 v4, s63, v4, v11
@@ -94938,27 +94949,27 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; GFX9-NEXT: v_mov_b32_e32 v5, s44
; GFX9-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_mov_b32_e32 v4, s56
+; GFX9-NEXT: v_mov_b32_e32 v4, s47
; GFX9-NEXT: v_perm_b32 v5, s45, v5, v11
-; GFX9-NEXT: v_perm_b32 v4, s73, v4, v11
+; GFX9-NEXT: v_perm_b32 v4, s74, v4, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; GFX9-NEXT: v_mov_b32_e32 v6, s41
; GFX9-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v5, s43
; GFX9-NEXT: v_perm_b32 v6, s42, v6, v11
-; GFX9-NEXT: v_perm_b32 v5, s74, v5, v11
+; GFX9-NEXT: v_perm_b32 v5, s75, v5, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; GFX9-NEXT: v_mov_b32_e32 v7, s14
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v6, s40
; GFX9-NEXT: v_perm_b32 v7, s15, v7, v11
-; GFX9-NEXT: v_perm_b32 v6, s75, v6, v11
+; GFX9-NEXT: v_perm_b32 v6, s76, v6, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; GFX9-NEXT: v_mov_b32_e32 v8, s11
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v7, s13
; GFX9-NEXT: v_perm_b32 v8, s12, v8, v11
-; GFX9-NEXT: v_perm_b32 v7, s76, v7, v11
+; GFX9-NEXT: v_perm_b32 v7, s73, v7, v11
; GFX9-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; GFX9-NEXT: v_mov_b32_e32 v10, 0x6050400
; GFX9-NEXT: v_mov_b32_e32 v9, s8
@@ -94976,8 +94987,8 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_sdwa v9, v12, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v12, s62
; GFX9-NEXT: v_mov_b32_e32 v13, s57
-; GFX9-NEXT: v_perm_b32 v12, s60, v12, v10
-; GFX9-NEXT: v_perm_b32 v10, s47, v13, v10
+; GFX9-NEXT: v_perm_b32 v12, s61, v12, v10
+; GFX9-NEXT: v_perm_b32 v10, s56, v13, v10
; GFX9-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; GFX9-NEXT: s_mov_b32 s4, 0x6050400
; GFX9-NEXT: v_or_b32_sdwa v10, v10, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -95042,10 +95053,10 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v19, 0xc0c0004
; GFX9-NEXT: v_perm_b32 v16, v0, v16, s4
; GFX9-NEXT: s_add_i32 s62, s62, 3
-; GFX9-NEXT: v_mov_b32_e32 v0, s60
+; GFX9-NEXT: v_mov_b32_e32 v0, s61
; GFX9-NEXT: v_perm_b32 v17, s62, v0, v19
; GFX9-NEXT: s_add_i32 s57, s57, 3
-; GFX9-NEXT: v_mov_b32_e32 v0, s47
+; GFX9-NEXT: v_mov_b32_e32 v0, s56
; GFX9-NEXT: v_perm_b32 v20, s57, v0, v19
; GFX9-NEXT: s_add_i32 s59, s59, 3
; GFX9-NEXT: v_mov_b32_e32 v0, s58
@@ -95081,44 +95092,44 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; GFX9-NEXT: v_or_b32_sdwa v2, v3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v3, s29
; GFX9-NEXT: s_add_i32 s63, s63, 3
-; GFX9-NEXT: v_mov_b32_e32 v4, s61
+; GFX9-NEXT: v_mov_b32_e32 v4, s60
; GFX9-NEXT: v_perm_b32 v3, s28, v3, v19
; GFX9-NEXT: v_perm_b32 v4, s63, v4, v19
; GFX9-NEXT: v_add_u32_e32 v3, 0x300, v3
; GFX9-NEXT: v_add_u32_sdwa v4, v4, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s73, s73, 3
+; GFX9-NEXT: s_add_i32 s74, s74, 3
; GFX9-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_mov_b32_e32 v4, s56
+; GFX9-NEXT: v_mov_b32_e32 v4, s47
; GFX9-NEXT: s_add_i32 s45, s45, 3
; GFX9-NEXT: v_mov_b32_e32 v5, s44
-; GFX9-NEXT: v_perm_b32 v4, s73, v4, v19
+; GFX9-NEXT: v_perm_b32 v4, s74, v4, v19
; GFX9-NEXT: v_perm_b32 v5, s45, v5, v19
; GFX9-NEXT: v_add_u32_e32 v4, 0x300, v4
; GFX9-NEXT: v_add_u32_sdwa v5, v5, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s74, s74, 3
+; GFX9-NEXT: s_add_i32 s75, s75, 3
; GFX9-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v5, s43
; GFX9-NEXT: s_add_i32 s42, s42, 3
; GFX9-NEXT: v_mov_b32_e32 v6, s41
-; GFX9-NEXT: v_perm_b32 v5, s74, v5, v19
+; GFX9-NEXT: v_perm_b32 v5, s75, v5, v19
; GFX9-NEXT: v_perm_b32 v6, s42, v6, v19
; GFX9-NEXT: v_add_u32_e32 v5, 0x300, v5
; GFX9-NEXT: v_add_u32_sdwa v6, v6, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s75, s75, 3
+; GFX9-NEXT: s_add_i32 s76, s76, 3
; GFX9-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v6, s40
; GFX9-NEXT: s_add_i32 s15, s15, 3
; GFX9-NEXT: v_mov_b32_e32 v7, s14
-; GFX9-NEXT: v_perm_b32 v6, s75, v6, v19
+; GFX9-NEXT: v_perm_b32 v6, s76, v6, v19
; GFX9-NEXT: v_perm_b32 v7, s15, v7, v19
; GFX9-NEXT: v_add_u32_e32 v6, 0x300, v6
; GFX9-NEXT: v_add_u32_sdwa v7, v7, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; GFX9-NEXT: s_add_i32 s76, s76, 3
+; GFX9-NEXT: s_add_i32 s73, s73, 3
; GFX9-NEXT: v_or_b32_sdwa v6, v7, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_mov_b32_e32 v7, s13
; GFX9-NEXT: s_add_i32 s12, s12, 3
; GFX9-NEXT: v_mov_b32_e32 v22, s11
-; GFX9-NEXT: v_perm_b32 v7, s76, v7, v19
+; GFX9-NEXT: v_perm_b32 v7, s73, v7, v19
; GFX9-NEXT: v_perm_b32 v22, s12, v22, v19
; GFX9-NEXT: v_add_u32_e32 v7, 0x300, v7
; GFX9-NEXT: v_add_u32_sdwa v22, v22, s5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.576bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.576bit.ll
index 8c1a872985896..72d0994bad4d6 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.576bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.576bit.ll
@@ -10794,15 +10794,15 @@ define inreg <36 x i16> @bitcast_v18f32_to_v36i16_scalar(<18 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[25:26], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[26:27], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v1
; SI-NEXT: s_branch .LBB29_6
; SI-NEXT: .LBB29_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -10823,15 +10823,15 @@ define inreg <36 x i16> @bitcast_v18f32_to_v36i16_scalar(<18 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v15, s7
; SI-NEXT: v_mov_b32_e32 v16, s4
; SI-NEXT: v_mov_b32_e32 v17, s5
-; SI-NEXT: v_mov_b32_e32 v36, s60
-; SI-NEXT: v_mov_b32_e32 v35, s61
-; SI-NEXT: v_mov_b32_e32 v34, s62
-; SI-NEXT: v_mov_b32_e32 v33, s63
-; SI-NEXT: v_mov_b32_e32 v32, s72
-; SI-NEXT: v_mov_b32_e32 v31, s73
-; SI-NEXT: v_mov_b32_e32 v30, s74
-; SI-NEXT: v_mov_b32_e32 v29, s75
-; SI-NEXT: v_mov_b32_e32 v28, s76
+; SI-NEXT: v_mov_b32_e32 v35, s60
+; SI-NEXT: v_mov_b32_e32 v34, s61
+; SI-NEXT: v_mov_b32_e32 v33, s62
+; SI-NEXT: v_mov_b32_e32 v32, s63
+; SI-NEXT: v_mov_b32_e32 v31, s72
+; SI-NEXT: v_mov_b32_e32 v30, s73
+; SI-NEXT: v_mov_b32_e32 v29, s74
+; SI-NEXT: v_mov_b32_e32 v28, s75
+; SI-NEXT: v_mov_b32_e32 v27, s76
; SI-NEXT: v_mov_b32_e32 v18, s8
; SI-NEXT: v_mov_b32_e32 v19, s10
; SI-NEXT: v_mov_b32_e32 v20, s12
@@ -10852,41 +10852,41 @@ define inreg <36 x i16> @bitcast_v18f32_to_v36i16_scalar(<18 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_or_b32_e32 v0, v0, v26
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v35
; SI-NEXT: v_or_b32_e32 v2, v2, v25
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v34
; SI-NEXT: v_or_b32_e32 v4, v4, v24
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v33
; SI-NEXT: v_or_b32_e32 v6, v6, v23
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
; SI-NEXT: v_or_b32_e32 v8, v8, v22
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v31
; SI-NEXT: v_or_b32_e32 v10, v10, v21
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v30
; SI-NEXT: v_or_b32_e32 v12, v12, v20
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v29
; SI-NEXT: v_or_b32_e32 v14, v14, v19
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v28
; SI-NEXT: v_or_b32_e32 v16, v16, v18
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v27
; SI-NEXT: v_or_b32_e32 v1, v1, v26
; SI-NEXT: v_or_b32_e32 v3, v3, v25
; SI-NEXT: v_or_b32_e32 v5, v5, v24
@@ -13607,15 +13607,15 @@ define inreg <36 x half> @bitcast_v18f32_to_v36f16_scalar(<18 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[25:26], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[26:27], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v1
; SI-NEXT: s_branch .LBB33_6
; SI-NEXT: .LBB33_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -13636,15 +13636,15 @@ define inreg <36 x half> @bitcast_v18f32_to_v36f16_scalar(<18 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v15, s7
; SI-NEXT: v_mov_b32_e32 v16, s4
; SI-NEXT: v_mov_b32_e32 v17, s5
-; SI-NEXT: v_mov_b32_e32 v36, s60
-; SI-NEXT: v_mov_b32_e32 v35, s61
-; SI-NEXT: v_mov_b32_e32 v34, s62
-; SI-NEXT: v_mov_b32_e32 v33, s63
-; SI-NEXT: v_mov_b32_e32 v32, s72
-; SI-NEXT: v_mov_b32_e32 v31, s73
-; SI-NEXT: v_mov_b32_e32 v30, s74
-; SI-NEXT: v_mov_b32_e32 v29, s75
-; SI-NEXT: v_mov_b32_e32 v28, s76
+; SI-NEXT: v_mov_b32_e32 v35, s60
+; SI-NEXT: v_mov_b32_e32 v34, s61
+; SI-NEXT: v_mov_b32_e32 v33, s62
+; SI-NEXT: v_mov_b32_e32 v32, s63
+; SI-NEXT: v_mov_b32_e32 v31, s72
+; SI-NEXT: v_mov_b32_e32 v30, s73
+; SI-NEXT: v_mov_b32_e32 v29, s74
+; SI-NEXT: v_mov_b32_e32 v28, s75
+; SI-NEXT: v_mov_b32_e32 v27, s76
; SI-NEXT: v_mov_b32_e32 v18, s8
; SI-NEXT: v_mov_b32_e32 v19, s10
; SI-NEXT: v_mov_b32_e32 v20, s12
@@ -13665,41 +13665,41 @@ define inreg <36 x half> @bitcast_v18f32_to_v36f16_scalar(<18 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
; SI-NEXT: v_or_b32_e32 v0, v0, v26
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v35
; SI-NEXT: v_or_b32_e32 v2, v2, v25
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v34
; SI-NEXT: v_or_b32_e32 v4, v4, v24
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v33
; SI-NEXT: v_or_b32_e32 v6, v6, v23
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
; SI-NEXT: v_or_b32_e32 v8, v8, v22
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v31
; SI-NEXT: v_or_b32_e32 v10, v10, v21
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v30
; SI-NEXT: v_or_b32_e32 v12, v12, v20
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v29
; SI-NEXT: v_or_b32_e32 v14, v14, v19
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v28
; SI-NEXT: v_or_b32_e32 v16, v16, v18
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
-; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v27
; SI-NEXT: v_or_b32_e32 v1, v1, v26
; SI-NEXT: v_or_b32_e32 v3, v3, v25
; SI-NEXT: v_or_b32_e32 v5, v5, v24
@@ -22291,20 +22291,20 @@ define <36 x i16> @bitcast_v9f64_to_v36i16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v18
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr34
-; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr21
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr19
-; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
@@ -22317,13 +22317,13 @@ define <36 x i16> @bitcast_v9f64_to_v36i16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v21, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v22, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v23, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v26, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v29, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v25, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v28, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v31, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v5
@@ -22348,13 +22348,13 @@ define <36 x i16> @bitcast_v9f64_to_v36i16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v21, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v22, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v23, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v26, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v29, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v25, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v28, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v31, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v5
@@ -22364,16 +22364,16 @@ define <36 x i16> @bitcast_v9f64_to_v36i16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
@@ -22383,12 +22383,12 @@ define <36 x i16> @bitcast_v9f64_to_v36i16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v0, v0, v31
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v35
-; SI-NEXT: v_or_b32_e32 v2, v2, v29
+; SI-NEXT: v_or_b32_e32 v2, v2, v28
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v34
-; SI-NEXT: v_or_b32_e32 v4, v4, v26
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v34
+; SI-NEXT: v_or_b32_e32 v4, v4, v25
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v33
; SI-NEXT: v_or_b32_e32 v6, v6, v23
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
@@ -22397,19 +22397,19 @@ define <36 x i16> @bitcast_v9f64_to_v36i16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v30
; SI-NEXT: v_or_b32_e32 v10, v10, v21
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v12, v12, v20
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
; SI-NEXT: v_or_b32_e32 v14, v14, v19
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v26
; SI-NEXT: v_or_b32_e32 v16, v16, v18
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v24
; SI-NEXT: v_or_b32_e32 v1, v1, v31
-; SI-NEXT: v_or_b32_e32 v3, v3, v29
-; SI-NEXT: v_or_b32_e32 v5, v5, v26
+; SI-NEXT: v_or_b32_e32 v3, v3, v28
+; SI-NEXT: v_or_b32_e32 v5, v5, v25
; SI-NEXT: v_or_b32_e32 v7, v7, v23
; SI-NEXT: v_or_b32_e32 v9, v9, v22
; SI-NEXT: v_or_b32_e32 v11, v11, v21
@@ -23021,12 +23021,12 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; VI-NEXT: v_lshrrev_b32_e32 v19, 16, v14
; VI-NEXT: v_lshrrev_b32_e32 v24, 16, v13
; VI-NEXT: v_lshrrev_b32_e32 v21, 16, v12
-; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v26, 16, v11
; VI-NEXT: v_lshrrev_b32_e32 v23, 16, v10
-; VI-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v8
; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v7
-; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v6
; VI-NEXT: v_lshrrev_b32_e32 v32, 16, v5
; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v4
; VI-NEXT: v_lshrrev_b32_e32 v34, 16, v3
@@ -23056,8 +23056,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v33, s59
; VI-NEXT: v_mov_b32_e32 v31, s58
; VI-NEXT: v_mov_b32_e32 v29, s57
-; VI-NEXT: v_mov_b32_e32 v28, s56
-; VI-NEXT: v_mov_b32_e32 v27, s47
+; VI-NEXT: v_mov_b32_e32 v27, s56
+; VI-NEXT: v_mov_b32_e32 v25, s47
; VI-NEXT: v_mov_b32_e32 v23, s46
; VI-NEXT: v_mov_b32_e32 v21, s45
; VI-NEXT: v_mov_b32_e32 v19, s44
@@ -23066,8 +23066,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v34, s41
; VI-NEXT: v_mov_b32_e32 v32, s40
; VI-NEXT: v_mov_b32_e32 v30, s15
-; VI-NEXT: v_mov_b32_e32 v26, s14
-; VI-NEXT: v_mov_b32_e32 v25, s13
+; VI-NEXT: v_mov_b32_e32 v28, s14
+; VI-NEXT: v_mov_b32_e32 v26, s13
; VI-NEXT: v_mov_b32_e32 v24, s12
; VI-NEXT: v_mov_b32_e32 v22, s11
; VI-NEXT: v_mov_b32_e32 v20, s10
@@ -23075,7 +23075,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
-; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
@@ -23086,12 +23087,12 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v34
; VI-NEXT: v_or_b32_sdwa v4, v4, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v32
-; VI-NEXT: v_or_b32_sdwa v6, v6, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v30
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
-; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; VI-NEXT: v_or_b32_sdwa v6, v6, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v30
+; VI-NEXT: v_or_b32_sdwa v8, v8, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; VI-NEXT: v_or_b32_sdwa v10, v10, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v25
+; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v26
; VI-NEXT: v_or_b32_sdwa v12, v12, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v24
; VI-NEXT: v_or_b32_sdwa v14, v14, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -23101,9 +23102,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; VI-NEXT: v_or_b32_sdwa v1, v1, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v3, v3, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v5, v5, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v7, v7, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v8, v8, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v9, v9, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v7, v7, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v9, v9, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v11, v11, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v13, v13, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v15, v15, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -23182,12 +23182,12 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; GFX9-NEXT: v_lshrrev_b32_e32 v19, 16, v14
; GFX9-NEXT: v_lshrrev_b32_e32 v24, 16, v13
; GFX9-NEXT: v_lshrrev_b32_e32 v21, 16, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v25, 16, v11
+; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v11
; GFX9-NEXT: v_lshrrev_b32_e32 v23, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v27, 16, v8
+; GFX9-NEXT: v_lshrrev_b32_e32 v28, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v25, 16, v8
; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v28, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v27, 16, v6
; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v5
; GFX9-NEXT: v_lshrrev_b32_e32 v29, 16, v4
; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v3
@@ -23217,8 +23217,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; GFX9-NEXT: v_mov_b32_e32 v33, s59
; GFX9-NEXT: v_mov_b32_e32 v31, s58
; GFX9-NEXT: v_mov_b32_e32 v29, s57
-; GFX9-NEXT: v_mov_b32_e32 v28, s56
-; GFX9-NEXT: v_mov_b32_e32 v27, s47
+; GFX9-NEXT: v_mov_b32_e32 v27, s56
+; GFX9-NEXT: v_mov_b32_e32 v25, s47
; GFX9-NEXT: v_mov_b32_e32 v23, s46
; GFX9-NEXT: v_mov_b32_e32 v21, s45
; GFX9-NEXT: v_mov_b32_e32 v19, s44
@@ -23227,8 +23227,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; GFX9-NEXT: v_mov_b32_e32 v34, s41
; GFX9-NEXT: v_mov_b32_e32 v32, s40
; GFX9-NEXT: v_mov_b32_e32 v30, s15
-; GFX9-NEXT: v_mov_b32_e32 v26, s14
-; GFX9-NEXT: v_mov_b32_e32 v25, s13
+; GFX9-NEXT: v_mov_b32_e32 v28, s14
+; GFX9-NEXT: v_mov_b32_e32 v26, s13
; GFX9-NEXT: v_mov_b32_e32 v24, s12
; GFX9-NEXT: v_mov_b32_e32 v22, s11
; GFX9-NEXT: v_mov_b32_e32 v20, s10
@@ -23236,7 +23236,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v29
-; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; GFX9-NEXT: v_lshlrev_b32_e32 v19, 16, v19
@@ -23247,12 +23248,12 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v34
; GFX9-NEXT: v_or_b32_sdwa v4, v29, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v32
-; GFX9-NEXT: v_or_b32_sdwa v6, v28, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v30
-; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v27
-; GFX9-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; GFX9-NEXT: v_or_b32_sdwa v6, v27, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v30
+; GFX9-NEXT: v_or_b32_sdwa v8, v25, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; GFX9-NEXT: v_or_b32_sdwa v10, v23, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v25
+; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v26
; GFX9-NEXT: v_or_b32_sdwa v12, v21, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v24
; GFX9-NEXT: v_or_b32_sdwa v14, v19, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -23262,9 +23263,8 @@ define inreg <36 x i16> @bitcast_v9f64_to_v36i16_scalar(<9 x double> inreg %a, i
; GFX9-NEXT: v_or_b32_sdwa v1, v33, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v3, v31, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v5, v29, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v7, v28, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v8, v27, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v9, v26, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v7, v27, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v9, v25, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v11, v23, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v13, v21, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v15, v19, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -25032,20 +25032,20 @@ define <36 x half> @bitcast_v9f64_to_v36f16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v18
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr35
-; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr34
-; SI-NEXT: ; implicit-def: $vgpr26
+; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr23
; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr21
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr19
-; SI-NEXT: ; implicit-def: $vgpr25
+; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr18
; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
@@ -25058,13 +25058,13 @@ define <36 x half> @bitcast_v9f64_to_v36f16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v21, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v22, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v23, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v26, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v29, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v25, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v28, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v31, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v5
@@ -25089,13 +25089,13 @@ define <36 x half> @bitcast_v9f64_to_v36f16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v21, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v22, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v23, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v26, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v29, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v25, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v28, v3, v2, 16
; SI-NEXT: v_alignbit_b32 v31, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v5
@@ -25105,16 +25105,16 @@ define <36 x half> @bitcast_v9f64_to_v36f16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
@@ -25124,12 +25124,12 @@ define <36 x half> @bitcast_v9f64_to_v36f16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_or_b32_e32 v0, v0, v31
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v35
-; SI-NEXT: v_or_b32_e32 v2, v2, v29
+; SI-NEXT: v_or_b32_e32 v2, v2, v28
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v34
-; SI-NEXT: v_or_b32_e32 v4, v4, v26
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v34
+; SI-NEXT: v_or_b32_e32 v4, v4, v25
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v33
; SI-NEXT: v_or_b32_e32 v6, v6, v23
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
@@ -25138,19 +25138,19 @@ define <36 x half> @bitcast_v9f64_to_v36f16(<9 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v30
; SI-NEXT: v_or_b32_e32 v10, v10, v21
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v12, v12, v20
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
; SI-NEXT: v_or_b32_e32 v14, v14, v19
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v26
; SI-NEXT: v_or_b32_e32 v16, v16, v18
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v24
; SI-NEXT: v_or_b32_e32 v1, v1, v31
-; SI-NEXT: v_or_b32_e32 v3, v3, v29
-; SI-NEXT: v_or_b32_e32 v5, v5, v26
+; SI-NEXT: v_or_b32_e32 v3, v3, v28
+; SI-NEXT: v_or_b32_e32 v5, v5, v25
; SI-NEXT: v_or_b32_e32 v7, v7, v23
; SI-NEXT: v_or_b32_e32 v9, v9, v22
; SI-NEXT: v_or_b32_e32 v11, v11, v21
@@ -25762,12 +25762,12 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; VI-NEXT: v_lshrrev_b32_e32 v19, 16, v14
; VI-NEXT: v_lshrrev_b32_e32 v24, 16, v13
; VI-NEXT: v_lshrrev_b32_e32 v21, 16, v12
-; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v26, 16, v11
; VI-NEXT: v_lshrrev_b32_e32 v23, 16, v10
-; VI-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v8
; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v7
-; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v6
; VI-NEXT: v_lshrrev_b32_e32 v32, 16, v5
; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v4
; VI-NEXT: v_lshrrev_b32_e32 v34, 16, v3
@@ -25797,8 +25797,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; VI-NEXT: v_mov_b32_e32 v33, s59
; VI-NEXT: v_mov_b32_e32 v31, s58
; VI-NEXT: v_mov_b32_e32 v29, s57
-; VI-NEXT: v_mov_b32_e32 v28, s56
-; VI-NEXT: v_mov_b32_e32 v27, s47
+; VI-NEXT: v_mov_b32_e32 v27, s56
+; VI-NEXT: v_mov_b32_e32 v25, s47
; VI-NEXT: v_mov_b32_e32 v23, s46
; VI-NEXT: v_mov_b32_e32 v21, s45
; VI-NEXT: v_mov_b32_e32 v19, s44
@@ -25807,8 +25807,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; VI-NEXT: v_mov_b32_e32 v34, s41
; VI-NEXT: v_mov_b32_e32 v32, s40
; VI-NEXT: v_mov_b32_e32 v30, s15
-; VI-NEXT: v_mov_b32_e32 v26, s14
-; VI-NEXT: v_mov_b32_e32 v25, s13
+; VI-NEXT: v_mov_b32_e32 v28, s14
+; VI-NEXT: v_mov_b32_e32 v26, s13
; VI-NEXT: v_mov_b32_e32 v24, s12
; VI-NEXT: v_mov_b32_e32 v22, s11
; VI-NEXT: v_mov_b32_e32 v20, s10
@@ -25816,7 +25816,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
-; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; VI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
@@ -25827,12 +25828,12 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v34
; VI-NEXT: v_or_b32_sdwa v4, v4, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v32
-; VI-NEXT: v_or_b32_sdwa v6, v6, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v28, 16, v30
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
-; VI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; VI-NEXT: v_or_b32_sdwa v6, v6, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v30
+; VI-NEXT: v_or_b32_sdwa v8, v8, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; VI-NEXT: v_or_b32_sdwa v10, v10, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v25
+; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v26
; VI-NEXT: v_or_b32_sdwa v12, v12, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v21, 16, v24
; VI-NEXT: v_or_b32_sdwa v14, v14, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -25842,9 +25843,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; VI-NEXT: v_or_b32_sdwa v1, v1, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v3, v3, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v5, v5, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v7, v7, v28 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v8, v8, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v9, v9, v26 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v7, v7, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v9, v9, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v11, v11, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v13, v13, v21 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v15, v15, v19 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -25923,12 +25923,12 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; GFX9-NEXT: v_lshrrev_b32_e32 v19, 16, v14
; GFX9-NEXT: v_lshrrev_b32_e32 v24, 16, v13
; GFX9-NEXT: v_lshrrev_b32_e32 v21, 16, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v25, 16, v11
+; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v11
; GFX9-NEXT: v_lshrrev_b32_e32 v23, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v26, 16, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v27, 16, v8
+; GFX9-NEXT: v_lshrrev_b32_e32 v28, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v25, 16, v8
; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v28, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v27, 16, v6
; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v5
; GFX9-NEXT: v_lshrrev_b32_e32 v29, 16, v4
; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v3
@@ -25958,8 +25958,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v33, s59
; GFX9-NEXT: v_mov_b32_e32 v31, s58
; GFX9-NEXT: v_mov_b32_e32 v29, s57
-; GFX9-NEXT: v_mov_b32_e32 v28, s56
-; GFX9-NEXT: v_mov_b32_e32 v27, s47
+; GFX9-NEXT: v_mov_b32_e32 v27, s56
+; GFX9-NEXT: v_mov_b32_e32 v25, s47
; GFX9-NEXT: v_mov_b32_e32 v23, s46
; GFX9-NEXT: v_mov_b32_e32 v21, s45
; GFX9-NEXT: v_mov_b32_e32 v19, s44
@@ -25968,8 +25968,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v34, s41
; GFX9-NEXT: v_mov_b32_e32 v32, s40
; GFX9-NEXT: v_mov_b32_e32 v30, s15
-; GFX9-NEXT: v_mov_b32_e32 v26, s14
-; GFX9-NEXT: v_mov_b32_e32 v25, s13
+; GFX9-NEXT: v_mov_b32_e32 v28, s14
+; GFX9-NEXT: v_mov_b32_e32 v26, s13
; GFX9-NEXT: v_mov_b32_e32 v24, s12
; GFX9-NEXT: v_mov_b32_e32 v22, s11
; GFX9-NEXT: v_mov_b32_e32 v20, s10
@@ -25977,7 +25977,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v29
-; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; GFX9-NEXT: v_lshlrev_b32_e32 v19, 16, v19
@@ -25988,12 +25989,12 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v34
; GFX9-NEXT: v_or_b32_sdwa v4, v29, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v32
-; GFX9-NEXT: v_or_b32_sdwa v6, v28, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v28, 16, v30
-; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v27
-; GFX9-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; GFX9-NEXT: v_or_b32_sdwa v6, v27, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v30
+; GFX9-NEXT: v_or_b32_sdwa v8, v25, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; GFX9-NEXT: v_or_b32_sdwa v10, v23, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v25
+; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v26
; GFX9-NEXT: v_or_b32_sdwa v12, v21, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v21, 16, v24
; GFX9-NEXT: v_or_b32_sdwa v14, v19, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -26003,9 +26004,8 @@ define inreg <36 x half> @bitcast_v9f64_to_v36f16_scalar(<9 x double> inreg %a,
; GFX9-NEXT: v_or_b32_sdwa v1, v33, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v3, v31, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v5, v29, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v7, v28, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v8, v27, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v9, v26, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v7, v27, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v9, v25, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v11, v23, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v13, v21, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v15, v19, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -30005,6 +30005,7 @@ define <36 x i16> @bitcast_v36f16_to_v36i16(<36 x half> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v16, v34, v16, s6
; GFX9-NEXT: v_perm_b32 v15, v33, v15, s6
; GFX9-NEXT: v_perm_b32 v14, v32, v14, s6
+; GFX9-NEXT: s_movk_i32 s7, 0x200
; GFX9-NEXT: v_perm_b32 v13, v31, v13, s6
; GFX9-NEXT: v_perm_b32 v12, v30, v12, s6
; GFX9-NEXT: v_perm_b32 v11, v29, v11, s6
@@ -30012,7 +30013,6 @@ define <36 x i16> @bitcast_v36f16_to_v36i16(<36 x half> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v9, v27, v9, s6
; GFX9-NEXT: v_perm_b32 v8, v26, v8, s6
; GFX9-NEXT: v_perm_b32 v7, v25, v7, s6
-; GFX9-NEXT: s_movk_i32 s7, 0x200
; GFX9-NEXT: v_perm_b32 v6, v24, v6, s6
; GFX9-NEXT: v_perm_b32 v5, v22, v5, s6
; GFX9-NEXT: v_perm_b32 v4, v23, v4, s6
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
index a2d89fb22315e..91eee5f461967 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
@@ -11809,16 +11809,16 @@ define inreg <40 x i16> @bitcast_v20f32_to_v40i16_scalar(<20 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[28:29], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[29:30], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v1
; SI-NEXT: s_branch .LBB29_6
; SI-NEXT: .LBB29_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -11841,16 +11841,16 @@ define inreg <40 x i16> @bitcast_v20f32_to_v40i16_scalar(<20 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v17, s7
; SI-NEXT: v_mov_b32_e32 v18, s4
; SI-NEXT: v_mov_b32_e32 v19, s5
-; SI-NEXT: v_mov_b32_e32 v48, s72
-; SI-NEXT: v_mov_b32_e32 v39, s73
-; SI-NEXT: v_mov_b32_e32 v38, s74
-; SI-NEXT: v_mov_b32_e32 v37, s75
-; SI-NEXT: v_mov_b32_e32 v36, s76
-; SI-NEXT: v_mov_b32_e32 v35, s77
-; SI-NEXT: v_mov_b32_e32 v34, s78
-; SI-NEXT: v_mov_b32_e32 v33, s79
-; SI-NEXT: v_mov_b32_e32 v32, s88
-; SI-NEXT: v_mov_b32_e32 v31, s89
+; SI-NEXT: v_mov_b32_e32 v39, s72
+; SI-NEXT: v_mov_b32_e32 v38, s73
+; SI-NEXT: v_mov_b32_e32 v37, s74
+; SI-NEXT: v_mov_b32_e32 v36, s75
+; SI-NEXT: v_mov_b32_e32 v35, s76
+; SI-NEXT: v_mov_b32_e32 v34, s77
+; SI-NEXT: v_mov_b32_e32 v33, s78
+; SI-NEXT: v_mov_b32_e32 v32, s79
+; SI-NEXT: v_mov_b32_e32 v31, s88
+; SI-NEXT: v_mov_b32_e32 v30, s89
; SI-NEXT: v_mov_b32_e32 v20, s10
; SI-NEXT: v_mov_b32_e32 v21, s12
; SI-NEXT: v_mov_b32_e32 v22, s14
@@ -11872,46 +11872,46 @@ define inreg <40 x i16> @bitcast_v20f32_to_v40i16_scalar(<20 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_or_b32_e32 v0, v0, v29
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v39
; SI-NEXT: v_or_b32_e32 v2, v2, v28
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v38
; SI-NEXT: v_or_b32_e32 v4, v4, v27
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v37
; SI-NEXT: v_or_b32_e32 v6, v6, v26
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
; SI-NEXT: v_or_b32_e32 v8, v8, v25
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v35
; SI-NEXT: v_or_b32_e32 v10, v10, v24
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v34
; SI-NEXT: v_or_b32_e32 v12, v12, v23
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v33
; SI-NEXT: v_or_b32_e32 v14, v14, v22
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v32
; SI-NEXT: v_or_b32_e32 v16, v16, v21
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v31
; SI-NEXT: v_or_b32_e32 v18, v18, v20
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v30
; SI-NEXT: v_or_b32_e32 v1, v1, v29
; SI-NEXT: v_or_b32_e32 v3, v3, v28
; SI-NEXT: v_or_b32_e32 v5, v5, v27
@@ -14968,16 +14968,16 @@ define inreg <40 x half> @bitcast_v20f32_to_v40f16_scalar(<20 x float> inreg %a,
; SI-NEXT: v_add_f32_e64 v0, s16, 1.0
; SI-NEXT: v_lshr_b64 v[28:29], v[2:3], 16
; SI-NEXT: v_lshr_b64 v[29:30], v[0:1], 16
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v3
-; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v1
+; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v7
+; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v1
; SI-NEXT: s_branch .LBB33_6
; SI-NEXT: .LBB33_5:
; SI-NEXT: v_mov_b32_e32 v0, s16
@@ -15000,16 +15000,16 @@ define inreg <40 x half> @bitcast_v20f32_to_v40f16_scalar(<20 x float> inreg %a,
; SI-NEXT: v_mov_b32_e32 v17, s7
; SI-NEXT: v_mov_b32_e32 v18, s4
; SI-NEXT: v_mov_b32_e32 v19, s5
-; SI-NEXT: v_mov_b32_e32 v48, s72
-; SI-NEXT: v_mov_b32_e32 v39, s73
-; SI-NEXT: v_mov_b32_e32 v38, s74
-; SI-NEXT: v_mov_b32_e32 v37, s75
-; SI-NEXT: v_mov_b32_e32 v36, s76
-; SI-NEXT: v_mov_b32_e32 v35, s77
-; SI-NEXT: v_mov_b32_e32 v34, s78
-; SI-NEXT: v_mov_b32_e32 v33, s79
-; SI-NEXT: v_mov_b32_e32 v32, s88
-; SI-NEXT: v_mov_b32_e32 v31, s89
+; SI-NEXT: v_mov_b32_e32 v39, s72
+; SI-NEXT: v_mov_b32_e32 v38, s73
+; SI-NEXT: v_mov_b32_e32 v37, s74
+; SI-NEXT: v_mov_b32_e32 v36, s75
+; SI-NEXT: v_mov_b32_e32 v35, s76
+; SI-NEXT: v_mov_b32_e32 v34, s77
+; SI-NEXT: v_mov_b32_e32 v33, s78
+; SI-NEXT: v_mov_b32_e32 v32, s79
+; SI-NEXT: v_mov_b32_e32 v31, s88
+; SI-NEXT: v_mov_b32_e32 v30, s89
; SI-NEXT: v_mov_b32_e32 v20, s10
; SI-NEXT: v_mov_b32_e32 v21, s12
; SI-NEXT: v_mov_b32_e32 v22, s14
@@ -15031,46 +15031,46 @@ define inreg <40 x half> @bitcast_v20f32_to_v40f16_scalar(<20 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
; SI-NEXT: v_or_b32_e32 v0, v0, v29
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v48
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v39
; SI-NEXT: v_or_b32_e32 v2, v2, v28
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v38
; SI-NEXT: v_or_b32_e32 v4, v4, v27
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v37
; SI-NEXT: v_or_b32_e32 v6, v6, v26
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
; SI-NEXT: v_or_b32_e32 v8, v8, v25
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v35
; SI-NEXT: v_or_b32_e32 v10, v10, v24
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v34
; SI-NEXT: v_or_b32_e32 v12, v12, v23
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v33
; SI-NEXT: v_or_b32_e32 v14, v14, v22
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v33
+; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v32
; SI-NEXT: v_or_b32_e32 v16, v16, v21
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v31
; SI-NEXT: v_or_b32_e32 v18, v18, v20
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
-; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v30
; SI-NEXT: v_or_b32_e32 v1, v1, v29
; SI-NEXT: v_or_b32_e32 v3, v3, v28
; SI-NEXT: v_or_b32_e32 v5, v5, v27
@@ -24602,24 +24602,24 @@ define <40 x i16> @bitcast_v10f64_to_v40i16(<10 x double> %a, i32 %b) #0 {
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v20
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr36
; SI-NEXT: ; implicit-def: $vgpr25
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr23
-; SI-NEXT: ; implicit-def: $vgpr31
+; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr21
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
@@ -24633,15 +24633,15 @@ define <40 x i16> @bitcast_v10f64_to_v40i16(<10 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v24, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v25, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v26, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v29, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v32, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v35, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v28, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v31, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v34, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v17
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
@@ -24667,33 +24667,33 @@ define <40 x i16> @bitcast_v10f64_to_v40i16(<10 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v24, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v25, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v26, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v29, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v32, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v35, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v28, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v31, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v34, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v17
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v1
; SI-NEXT: .LBB48_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
@@ -24702,39 +24702,39 @@ define <40 x i16> @bitcast_v10f64_to_v40i16(<10 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
-; SI-NEXT: v_or_b32_e32 v0, v0, v35
+; SI-NEXT: v_or_b32_e32 v0, v0, v34
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v39
-; SI-NEXT: v_or_b32_e32 v2, v2, v32
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v39
+; SI-NEXT: v_or_b32_e32 v2, v2, v31
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v38
-; SI-NEXT: v_or_b32_e32 v4, v4, v29
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v38
+; SI-NEXT: v_or_b32_e32 v4, v4, v28
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v37
; SI-NEXT: v_or_b32_e32 v6, v6, v26
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
; SI-NEXT: v_or_b32_e32 v8, v8, v25
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v35
; SI-NEXT: v_or_b32_e32 v10, v10, v24
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v33
; SI-NEXT: v_or_b32_e32 v12, v12, v23
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
; SI-NEXT: v_or_b32_e32 v14, v14, v22
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v30
; SI-NEXT: v_or_b32_e32 v16, v16, v21
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v18, v18, v20
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
-; SI-NEXT: v_or_b32_e32 v1, v1, v35
-; SI-NEXT: v_or_b32_e32 v3, v3, v32
-; SI-NEXT: v_or_b32_e32 v5, v5, v29
+; SI-NEXT: v_or_b32_e32 v1, v1, v34
+; SI-NEXT: v_or_b32_e32 v3, v3, v31
+; SI-NEXT: v_or_b32_e32 v5, v5, v28
; SI-NEXT: v_or_b32_e32 v7, v7, v26
; SI-NEXT: v_or_b32_e32 v9, v9, v25
; SI-NEXT: v_or_b32_e32 v11, v11, v24
@@ -25405,12 +25405,12 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; VI-NEXT: v_lshrrev_b32_e32 v23, 16, v14
; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v13
; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v12
-; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v11
; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v10
-; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v9
-; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v32, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v8
; VI-NEXT: v_lshrrev_b32_e32 v34, 16, v7
-; VI-NEXT: v_lshrrev_b32_e32 v32, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v6
; VI-NEXT: v_lshrrev_b32_e32 v36, 16, v5
; VI-NEXT: v_lshrrev_b32_e32 v33, 16, v4
; VI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
@@ -25442,8 +25442,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; VI-NEXT: v_mov_b32_e32 v37, s63
; VI-NEXT: v_mov_b32_e32 v35, s62
; VI-NEXT: v_mov_b32_e32 v33, s61
-; VI-NEXT: v_mov_b32_e32 v32, s60
-; VI-NEXT: v_mov_b32_e32 v31, s59
+; VI-NEXT: v_mov_b32_e32 v31, s60
+; VI-NEXT: v_mov_b32_e32 v29, s59
; VI-NEXT: v_mov_b32_e32 v27, s58
; VI-NEXT: v_mov_b32_e32 v25, s57
; VI-NEXT: v_mov_b32_e32 v23, s56
@@ -25453,8 +25453,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; VI-NEXT: v_mov_b32_e32 v38, s44
; VI-NEXT: v_mov_b32_e32 v36, s43
; VI-NEXT: v_mov_b32_e32 v34, s42
-; VI-NEXT: v_mov_b32_e32 v30, s41
-; VI-NEXT: v_mov_b32_e32 v29, s40
+; VI-NEXT: v_mov_b32_e32 v32, s41
+; VI-NEXT: v_mov_b32_e32 v30, s40
; VI-NEXT: v_mov_b32_e32 v28, s15
; VI-NEXT: v_mov_b32_e32 v26, s14
; VI-NEXT: v_mov_b32_e32 v24, s13
@@ -25463,7 +25463,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; VI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
-; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
@@ -25475,12 +25476,12 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; VI-NEXT: v_or_b32_sdwa v4, v4, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v36
-; VI-NEXT: v_or_b32_sdwa v6, v6, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v34
-; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
-; VI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; VI-NEXT: v_or_b32_sdwa v6, v6, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v34
+; VI-NEXT: v_or_b32_sdwa v8, v8, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v32
; VI-NEXT: v_or_b32_sdwa v10, v10, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v29
+; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v30
; VI-NEXT: v_or_b32_sdwa v12, v12, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; VI-NEXT: v_or_b32_sdwa v14, v14, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -25492,9 +25493,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; VI-NEXT: v_or_b32_sdwa v1, v1, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v3, v3, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v5, v5, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v7, v7, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v8, v8, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v9, v9, v30 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v7, v7, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v9, v9, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v11, v11, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v13, v13, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v15, v15, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -25583,12 +25583,12 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; GFX9-NEXT: v_lshrrev_b32_e32 v23, 16, v14
; GFX9-NEXT: v_lshrrev_b32_e32 v28, 16, v13
; GFX9-NEXT: v_lshrrev_b32_e32 v25, 16, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v29, 16, v11
+; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v11
; GFX9-NEXT: v_lshrrev_b32_e32 v27, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v31, 16, v8
+; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v29, 16, v8
; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v31, 16, v6
; GFX9-NEXT: v_lshrrev_b32_e32 v36, 16, v5
; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v4
; GFX9-NEXT: v_lshrrev_b32_e32 v38, 16, v3
@@ -25620,8 +25620,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v37, s63
; GFX9-NEXT: v_mov_b32_e32 v35, s62
; GFX9-NEXT: v_mov_b32_e32 v33, s61
-; GFX9-NEXT: v_mov_b32_e32 v32, s60
-; GFX9-NEXT: v_mov_b32_e32 v31, s59
+; GFX9-NEXT: v_mov_b32_e32 v31, s60
+; GFX9-NEXT: v_mov_b32_e32 v29, s59
; GFX9-NEXT: v_mov_b32_e32 v27, s58
; GFX9-NEXT: v_mov_b32_e32 v25, s57
; GFX9-NEXT: v_mov_b32_e32 v23, s56
@@ -25631,8 +25631,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v38, s44
; GFX9-NEXT: v_mov_b32_e32 v36, s43
; GFX9-NEXT: v_mov_b32_e32 v34, s42
-; GFX9-NEXT: v_mov_b32_e32 v30, s41
-; GFX9-NEXT: v_mov_b32_e32 v29, s40
+; GFX9-NEXT: v_mov_b32_e32 v32, s41
+; GFX9-NEXT: v_mov_b32_e32 v30, s40
; GFX9-NEXT: v_mov_b32_e32 v28, s15
; GFX9-NEXT: v_mov_b32_e32 v26, s14
; GFX9-NEXT: v_mov_b32_e32 v24, s13
@@ -25641,7 +25641,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v33
-; GFX9-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
@@ -25653,12 +25654,12 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; GFX9-NEXT: v_or_b32_sdwa v4, v33, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v36
-; GFX9-NEXT: v_or_b32_sdwa v6, v32, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v32, 16, v34
-; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
-; GFX9-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; GFX9-NEXT: v_or_b32_sdwa v6, v31, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v34
+; GFX9-NEXT: v_or_b32_sdwa v8, v29, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v32
; GFX9-NEXT: v_or_b32_sdwa v10, v27, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v29
+; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v30
; GFX9-NEXT: v_or_b32_sdwa v12, v25, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; GFX9-NEXT: v_or_b32_sdwa v14, v23, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -25670,9 +25671,8 @@ define inreg <40 x i16> @bitcast_v10f64_to_v40i16_scalar(<10 x double> inreg %a,
; GFX9-NEXT: v_or_b32_sdwa v1, v37, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v3, v35, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v5, v33, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v7, v32, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v8, v31, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v9, v30, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v7, v31, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v9, v29, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v11, v27, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v13, v25, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v15, v23, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -27681,24 +27681,24 @@ define <40 x half> @bitcast_v10f64_to_v40f16(<10 x double> %a, i32 %b) #0 {
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v20
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr39
-; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr38
-; SI-NEXT: ; implicit-def: $vgpr29
+; SI-NEXT: ; implicit-def: $vgpr28
; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr26
; SI-NEXT: ; implicit-def: $vgpr36
; SI-NEXT: ; implicit-def: $vgpr25
-; SI-NEXT: ; implicit-def: $vgpr34
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: ; implicit-def: $vgpr24
; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: ; implicit-def: $vgpr23
-; SI-NEXT: ; implicit-def: $vgpr31
+; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr22
; SI-NEXT: ; implicit-def: $vgpr30
; SI-NEXT: ; implicit-def: $vgpr21
-; SI-NEXT: ; implicit-def: $vgpr28
+; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr20
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
@@ -27712,15 +27712,15 @@ define <40 x half> @bitcast_v10f64_to_v40f16(<10 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v24, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v25, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v26, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v29, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v32, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v35, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v28, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v31, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v34, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v17
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
@@ -27746,33 +27746,33 @@ define <40 x half> @bitcast_v10f64_to_v40f16(<10 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v24, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v25, v9, v8, 16
; SI-NEXT: v_alignbit_b32 v26, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v29, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v32, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v35, v1, v0, 16
+; SI-NEXT: v_alignbit_b32 v28, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v31, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v34, v1, v0, 16
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v19
-; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v17
+; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v17
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v15
-; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v13
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v13
; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v9
+; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v5
; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v1
; SI-NEXT: .LBB52_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
@@ -27781,39 +27781,39 @@ define <40 x half> @bitcast_v10f64_to_v40f16(<10 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
-; SI-NEXT: v_or_b32_e32 v0, v0, v35
+; SI-NEXT: v_or_b32_e32 v0, v0, v34
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v39
-; SI-NEXT: v_or_b32_e32 v2, v2, v32
+; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v39
+; SI-NEXT: v_or_b32_e32 v2, v2, v31
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v38
-; SI-NEXT: v_or_b32_e32 v4, v4, v29
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v38
+; SI-NEXT: v_or_b32_e32 v4, v4, v28
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v37
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v37
; SI-NEXT: v_or_b32_e32 v6, v6, v26
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
; SI-NEXT: v_or_b32_e32 v8, v8, v25
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
-; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v34
+; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v35
; SI-NEXT: v_or_b32_e32 v10, v10, v24
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v33
; SI-NEXT: v_or_b32_e32 v12, v12, v23
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v32
; SI-NEXT: v_or_b32_e32 v14, v14, v22
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v30
; SI-NEXT: v_or_b32_e32 v16, v16, v21
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
-; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v29
; SI-NEXT: v_or_b32_e32 v18, v18, v20
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v27
-; SI-NEXT: v_or_b32_e32 v1, v1, v35
-; SI-NEXT: v_or_b32_e32 v3, v3, v32
-; SI-NEXT: v_or_b32_e32 v5, v5, v29
+; SI-NEXT: v_or_b32_e32 v1, v1, v34
+; SI-NEXT: v_or_b32_e32 v3, v3, v31
+; SI-NEXT: v_or_b32_e32 v5, v5, v28
; SI-NEXT: v_or_b32_e32 v7, v7, v26
; SI-NEXT: v_or_b32_e32 v9, v9, v25
; SI-NEXT: v_or_b32_e32 v11, v11, v24
@@ -28484,12 +28484,12 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; VI-NEXT: v_lshrrev_b32_e32 v23, 16, v14
; VI-NEXT: v_lshrrev_b32_e32 v28, 16, v13
; VI-NEXT: v_lshrrev_b32_e32 v25, 16, v12
-; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v11
; VI-NEXT: v_lshrrev_b32_e32 v27, 16, v10
-; VI-NEXT: v_lshrrev_b32_e32 v30, 16, v9
-; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v32, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v29, 16, v8
; VI-NEXT: v_lshrrev_b32_e32 v34, 16, v7
-; VI-NEXT: v_lshrrev_b32_e32 v32, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v6
; VI-NEXT: v_lshrrev_b32_e32 v36, 16, v5
; VI-NEXT: v_lshrrev_b32_e32 v33, 16, v4
; VI-NEXT: v_lshrrev_b32_e32 v38, 16, v3
@@ -28521,8 +28521,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; VI-NEXT: v_mov_b32_e32 v37, s63
; VI-NEXT: v_mov_b32_e32 v35, s62
; VI-NEXT: v_mov_b32_e32 v33, s61
-; VI-NEXT: v_mov_b32_e32 v32, s60
-; VI-NEXT: v_mov_b32_e32 v31, s59
+; VI-NEXT: v_mov_b32_e32 v31, s60
+; VI-NEXT: v_mov_b32_e32 v29, s59
; VI-NEXT: v_mov_b32_e32 v27, s58
; VI-NEXT: v_mov_b32_e32 v25, s57
; VI-NEXT: v_mov_b32_e32 v23, s56
@@ -28532,8 +28532,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; VI-NEXT: v_mov_b32_e32 v38, s44
; VI-NEXT: v_mov_b32_e32 v36, s43
; VI-NEXT: v_mov_b32_e32 v34, s42
-; VI-NEXT: v_mov_b32_e32 v30, s41
-; VI-NEXT: v_mov_b32_e32 v29, s40
+; VI-NEXT: v_mov_b32_e32 v32, s41
+; VI-NEXT: v_mov_b32_e32 v30, s40
; VI-NEXT: v_mov_b32_e32 v28, s15
; VI-NEXT: v_mov_b32_e32 v26, s14
; VI-NEXT: v_mov_b32_e32 v24, s13
@@ -28542,7 +28542,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; VI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
-; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; VI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
@@ -28554,12 +28555,12 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; VI-NEXT: v_or_b32_sdwa v4, v4, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v36
-; VI-NEXT: v_or_b32_sdwa v6, v6, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v32, 16, v34
-; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
-; VI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; VI-NEXT: v_or_b32_sdwa v6, v6, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v34
+; VI-NEXT: v_or_b32_sdwa v8, v8, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v29, 16, v32
; VI-NEXT: v_or_b32_sdwa v10, v10, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v29
+; VI-NEXT: v_lshlrev_b32_e32 v27, 16, v30
; VI-NEXT: v_or_b32_sdwa v12, v12, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; VI-NEXT: v_or_b32_sdwa v14, v14, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -28571,9 +28572,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; VI-NEXT: v_or_b32_sdwa v1, v1, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v3, v3, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v5, v5, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v7, v7, v32 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v8, v8, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v9, v9, v30 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v7, v7, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v9, v9, v29 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v11, v11, v27 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v13, v13, v25 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v15, v15, v23 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -28662,12 +28662,12 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; GFX9-NEXT: v_lshrrev_b32_e32 v23, 16, v14
; GFX9-NEXT: v_lshrrev_b32_e32 v28, 16, v13
; GFX9-NEXT: v_lshrrev_b32_e32 v25, 16, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v29, 16, v11
+; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v11
; GFX9-NEXT: v_lshrrev_b32_e32 v27, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v30, 16, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v31, 16, v8
+; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v29, 16, v8
; GFX9-NEXT: v_lshrrev_b32_e32 v34, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v32, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v31, 16, v6
; GFX9-NEXT: v_lshrrev_b32_e32 v36, 16, v5
; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v4
; GFX9-NEXT: v_lshrrev_b32_e32 v38, 16, v3
@@ -28699,8 +28699,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; GFX9-NEXT: v_mov_b32_e32 v37, s63
; GFX9-NEXT: v_mov_b32_e32 v35, s62
; GFX9-NEXT: v_mov_b32_e32 v33, s61
-; GFX9-NEXT: v_mov_b32_e32 v32, s60
-; GFX9-NEXT: v_mov_b32_e32 v31, s59
+; GFX9-NEXT: v_mov_b32_e32 v31, s60
+; GFX9-NEXT: v_mov_b32_e32 v29, s59
; GFX9-NEXT: v_mov_b32_e32 v27, s58
; GFX9-NEXT: v_mov_b32_e32 v25, s57
; GFX9-NEXT: v_mov_b32_e32 v23, s56
@@ -28710,8 +28710,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; GFX9-NEXT: v_mov_b32_e32 v38, s44
; GFX9-NEXT: v_mov_b32_e32 v36, s43
; GFX9-NEXT: v_mov_b32_e32 v34, s42
-; GFX9-NEXT: v_mov_b32_e32 v30, s41
-; GFX9-NEXT: v_mov_b32_e32 v29, s40
+; GFX9-NEXT: v_mov_b32_e32 v32, s41
+; GFX9-NEXT: v_mov_b32_e32 v30, s40
; GFX9-NEXT: v_mov_b32_e32 v28, s15
; GFX9-NEXT: v_mov_b32_e32 v26, s14
; GFX9-NEXT: v_mov_b32_e32 v24, s13
@@ -28720,7 +28720,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; GFX9-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v33
-; GFX9-NEXT: v_lshlrev_b32_e32 v32, 16, v32
+; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; GFX9-NEXT: v_lshlrev_b32_e32 v23, 16, v23
@@ -28732,12 +28733,12 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; GFX9-NEXT: v_or_b32_sdwa v4, v33, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v36
-; GFX9-NEXT: v_or_b32_sdwa v6, v32, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v32, 16, v34
-; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
-; GFX9-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; GFX9-NEXT: v_or_b32_sdwa v6, v31, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v34
+; GFX9-NEXT: v_or_b32_sdwa v8, v29, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v29, 16, v32
; GFX9-NEXT: v_or_b32_sdwa v10, v27, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v29
+; GFX9-NEXT: v_lshlrev_b32_e32 v27, 16, v30
; GFX9-NEXT: v_or_b32_sdwa v12, v25, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v25, 16, v28
; GFX9-NEXT: v_or_b32_sdwa v14, v23, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -28749,9 +28750,8 @@ define inreg <40 x half> @bitcast_v10f64_to_v40f16_scalar(<10 x double> inreg %a
; GFX9-NEXT: v_or_b32_sdwa v1, v37, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v3, v35, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v5, v33, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v7, v32, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v8, v31, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v9, v30, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v7, v31, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v9, v29, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v11, v27, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v13, v25, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v15, v23, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -31760,20 +31760,18 @@ define inreg <40 x half> @bitcast_v40i16_to_v40f16_scalar(<40 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v20, s49, 7
; SI-NEXT: v_writelane_b32 v20, s30, 8
; SI-NEXT: v_writelane_b32 v20, s31, 9
-; SI-NEXT: v_readfirstlane_b32 s11, v1
-; SI-NEXT: ; implicit-def: $vgpr21 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s5, v5
+; SI-NEXT: v_readfirstlane_b32 s7, v5
; SI-NEXT: v_readfirstlane_b32 s49, v4
; SI-NEXT: v_readfirstlane_b32 s9, v3
; SI-NEXT: v_readfirstlane_b32 s48, v2
+; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s38, v0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v21, s11, 0
-; SI-NEXT: s_lshr_b32 s77, s29, 16
+; SI-NEXT: ; implicit-def: $vgpr21 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s75, s29, 16
; SI-NEXT: s_lshr_b32 s37, s28, 16
-; SI-NEXT: s_lshr_b32 s75, s27, 16
+; SI-NEXT: s_lshr_b32 s73, s27, 16
; SI-NEXT: s_lshr_b32 s36, s26, 16
-; SI-NEXT: s_lshr_b32 s73, s25, 16
+; SI-NEXT: s_lshr_b32 s63, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s23, 16
; SI-NEXT: s_lshr_b32 s34, s22, 16
@@ -31783,138 +31781,141 @@ define inreg <40 x half> @bitcast_v40i16_to_v40f16_scalar(<40 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s30, s18, 16
; SI-NEXT: s_lshr_b32 s94, s17, 16
; SI-NEXT: s_lshr_b32 s57, s16, 16
-; SI-NEXT: s_lshr_b32 s91, s5, 16
-; SI-NEXT: s_lshr_b32 s7, s49, 16
-; SI-NEXT: s_lshr_b32 s89, s9, 16
+; SI-NEXT: s_lshr_b32 s89, s7, 16
+; SI-NEXT: s_lshr_b32 s5, s49, 16
+; SI-NEXT: s_lshr_b32 s79, s9, 16
; SI-NEXT: s_lshr_b32 s39, s48, 16
-; SI-NEXT: s_lshr_b32 s79, s11, 16
+; SI-NEXT: s_lshr_b32 s77, s91, 16
; SI-NEXT: s_lshr_b32 s59, s38, 16
; SI-NEXT: v_readfirstlane_b32 s4, v6
-; SI-NEXT: v_writelane_b32 v21, s9, 1
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v21, s9, 0
; SI-NEXT: s_cmp_lg_u32 s4, 0
+; SI-NEXT: v_writelane_b32 v21, s7, 1
; SI-NEXT: v_writelane_b32 v21, s5, 2
-; SI-NEXT: v_writelane_b32 v21, s7, 3
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s15, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s94, 16
+; SI-NEXT: s_and_b32 s11, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s94, 16
; SI-NEXT: s_lshl_b32 s56, s57, 16
-; SI-NEXT: s_mov_b32 s13, s57
-; SI-NEXT: s_or_b32 s57, s15, s41
-; SI-NEXT: s_and_b32 s41, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s95, 16
-; SI-NEXT: s_lshl_b32 s46, s59, 16
-; SI-NEXT: s_mov_b32 s15, s59
-; SI-NEXT: s_or_b32 s59, s41, s43
-; SI-NEXT: s_and_b32 s41, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; SI-NEXT: s_mov_b32 s62, s57
+; SI-NEXT: s_or_b32 s57, s11, s13
+; SI-NEXT: s_and_b32 s11, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s95, 16
+; SI-NEXT: s_lshl_b32 s14, s59, 16
+; SI-NEXT: s_mov_b32 s72, s59
+; SI-NEXT: s_or_b32 s59, s11, s13
+; SI-NEXT: s_and_b32 s11, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; SI-NEXT: s_or_b32 s61, s11, s13
+; SI-NEXT: s_and_b32 s11, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; SI-NEXT: s_or_b32 s47, s11, s13
+; SI-NEXT: s_and_b32 s11, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s63, 16
+; SI-NEXT: s_or_b32 s45, s11, s13
+; SI-NEXT: s_and_b32 s11, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s73, 16
+; SI-NEXT: s_or_b32 s43, s11, s13
+; SI-NEXT: s_and_b32 s11, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s75, 16
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_or_b32 s61, s41, s43
-; SI-NEXT: s_and_b32 s41, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s56
+; SI-NEXT: s_or_b32 s41, s11, s13
+; SI-NEXT: s_and_b32 s11, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s77, 16
+; SI-NEXT: s_or_b32 s10, s4, s56
; SI-NEXT: s_and_b32 s4, s18, 0xffff
; SI-NEXT: s_lshl_b32 s58, s30, 16
-; SI-NEXT: s_or_b32 s63, s41, s43
-; SI-NEXT: s_and_b32 s41, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s73, 16
-; SI-NEXT: s_or_b32 s40, s4, s58
+; SI-NEXT: s_or_b32 s15, s11, s13
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s11, s79, 16
+; SI-NEXT: s_or_b32 s8, s4, s58
; SI-NEXT: s_and_b32 s4, s20, 0xffff
; SI-NEXT: s_lshl_b32 s60, s31, 16
-; SI-NEXT: s_mov_b32 s78, s73
-; SI-NEXT: s_or_b32 s73, s41, s43
-; SI-NEXT: s_and_b32 s41, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s75, 16
-; SI-NEXT: s_or_b32 s14, s4, s60
+; SI-NEXT: s_or_b32 s13, s9, s11
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s9, s89, 16
+; SI-NEXT: s_or_b32 s6, s4, s60
+; SI-NEXT: s_lshl_b32 s46, s34, 16
+; SI-NEXT: s_or_b32 s93, s7, s9
+; SI-NEXT: s_mov_b32 s11, s57
+; SI-NEXT: s_lshr_b64 s[56:57], s[56:57], 16
+; SI-NEXT: s_mov_b32 s7, s61
+; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s62, s34, 16
-; SI-NEXT: s_mov_b32 s88, s75
-; SI-NEXT: s_or_b32 s75, s41, s43
-; SI-NEXT: s_and_b32 s41, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s77, 16
-; SI-NEXT: s_or_b32 s12, s4, s62
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s72, s35, 16
-; SI-NEXT: s_mov_b32 s90, s77
-; SI-NEXT: s_or_b32 s77, s41, s43
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s79, 16
-; SI-NEXT: s_or_b32 s10, s4, s72
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s74, s36, 16
-; SI-NEXT: s_or_b32 s47, s11, s41
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s89, 16
-; SI-NEXT: s_mov_b32 s41, s59
+; SI-NEXT: s_lshl_b32 s44, s35, 16
+; SI-NEXT: s_mov_b32 s57, s62
+; SI-NEXT: s_mov_b32 s9, s59
; SI-NEXT: s_lshr_b64 s[58:59], s[58:59], 16
-; SI-NEXT: s_or_b32 s8, s4, s74
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s76, s37, 16
-; SI-NEXT: s_or_b32 s45, s9, s11
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s91, 16
-; SI-NEXT: s_mov_b32 s59, s15
-; SI-NEXT: s_mov_b32 s15, s61
-; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_mov_b32 s11, s73
-; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
-; SI-NEXT: s_or_b32 s6, s4, s76
-; SI-NEXT: s_and_b32 s4, s38, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s39, 16
-; SI-NEXT: s_or_b32 s93, s5, s9
-; SI-NEXT: s_mov_b32 s73, s78
-; SI-NEXT: s_mov_b32 s9, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
-; SI-NEXT: s_mov_b32 s61, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[46:47], 16
+; SI-NEXT: s_mov_b32 s61, s63
+; SI-NEXT: s_lshr_b64 s[62:63], s[46:47], 16
; SI-NEXT: s_or_b32 s4, s4, s46
-; SI-NEXT: s_lshl_b32 s92, s7, 16
-; SI-NEXT: s_mov_b32 s75, s88
-; SI-NEXT: s_mov_b32 s7, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
+; SI-NEXT: s_lshl_b32 s42, s36, 16
+; SI-NEXT: s_mov_b32 s59, s72
+; SI-NEXT: s_mov_b32 s63, s61
+; SI-NEXT: s_and_b32 s46, s24, 0xffff
+; SI-NEXT: s_mov_b32 s61, s73
+; SI-NEXT: s_lshr_b64 s[72:73], s[44:45], 16
+; SI-NEXT: s_lshl_b32 s40, s37, 16
+; SI-NEXT: s_or_b32 s46, s46, s44
+; SI-NEXT: s_mov_b32 s73, s61
+; SI-NEXT: s_and_b32 s44, s26, 0xffff
+; SI-NEXT: s_mov_b32 s61, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[42:43], 16
+; SI-NEXT: s_or_b32 s44, s44, s42
+; SI-NEXT: s_mov_b32 s75, s61
+; SI-NEXT: s_and_b32 s42, s28, 0xffff
+; SI-NEXT: s_mov_b32 s61, s77
+; SI-NEXT: s_lshr_b64 s[76:77], s[40:41], 16
+; SI-NEXT: s_lshl_b32 s12, s39, 16
+; SI-NEXT: s_or_b32 s42, s42, s40
+; SI-NEXT: s_mov_b32 s77, s61
+; SI-NEXT: s_and_b32 s40, s38, 0xffff
+; SI-NEXT: s_mov_b32 s61, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[14:15], 16
+; SI-NEXT: s_lshl_b32 s92, s5, 16
+; SI-NEXT: s_or_b32 s40, s40, s14
; SI-NEXT: s_mov_b32 s79, s61
-; SI-NEXT: s_and_b32 s46, s48, 0xffff
+; SI-NEXT: s_and_b32 s14, s48, 0xffff
; SI-NEXT: s_mov_b32 s61, s89
-; SI-NEXT: s_lshr_b64 s[88:89], s[44:45], 16
-; SI-NEXT: s_mov_b32 s43, s57
-; SI-NEXT: s_lshr_b64 s[56:57], s[56:57], 16
-; SI-NEXT: s_mov_b32 s77, s90
-; SI-NEXT: s_or_b32 s46, s46, s44
+; SI-NEXT: s_lshr_b64 s[88:89], s[12:13], 16
+; SI-NEXT: s_or_b32 s14, s14, s12
; SI-NEXT: s_mov_b32 s89, s61
-; SI-NEXT: s_and_b32 s44, s49, 0xffff
+; SI-NEXT: s_and_b32 s12, s49, 0xffff
; SI-NEXT: s_mov_b32 s61, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[92:93], 16
-; SI-NEXT: s_mov_b32 s57, s13
-; SI-NEXT: s_mov_b32 s13, s63
-; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
; SI-NEXT: s_mov_b32 s5, s47
; SI-NEXT: s_mov_b32 s47, s45
-; SI-NEXT: s_or_b32 s44, s44, s92
-; SI-NEXT: s_mov_b32 s45, s93
+; SI-NEXT: s_mov_b32 s45, s43
+; SI-NEXT: s_mov_b32 s43, s41
+; SI-NEXT: s_mov_b32 s41, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_or_b32 s12, s12, s92
+; SI-NEXT: s_mov_b32 s13, s93
; SI-NEXT: s_mov_b32 s91, s61
; SI-NEXT: s_mov_b64 s[92:93], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
; SI-NEXT: s_mov_b64 s[92:93], -1
-; SI-NEXT: ; implicit-def: $sgpr42
+; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; implicit-def: $sgpr8
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr46
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr44
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr44
+; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: .LBB57_3: ; %Flow
; SI-NEXT: s_and_b64 s[92:93], s[92:93], exec
@@ -31924,210 +31925,209 @@ define inreg <40 x half> @bitcast_v40i16_to_v40f16_scalar(<40 x i16> inreg %a, i
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s4, s49, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff
-; SI-NEXT: v_readlane_b32 s4, v21, 3
+; SI-NEXT: v_readlane_b32 s4, v21, 2
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: s_add_i32 s44, s4, 0x30000
-; SI-NEXT: v_readlane_b32 s4, v21, 2
+; SI-NEXT: s_add_i32 s12, s4, 0x30000
+; SI-NEXT: v_readlane_b32 s4, v21, 1
; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
+; SI-NEXT: s_lshl_b32 s5, s89, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s45, s4, 0x30000
+; SI-NEXT: s_add_i32 s13, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s48, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s39, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s46, s4, 0x30000
-; SI-NEXT: v_readlane_b32 s4, v21, 1
+; SI-NEXT: s_add_i32 s14, s4, 0x30000
+; SI-NEXT: v_readlane_b32 s4, v21, 0
; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
+; SI-NEXT: s_lshl_b32 s5, s79, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s47, s4, 0x30000
+; SI-NEXT: s_add_i32 s15, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s38, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s59, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v21, 0
-; SI-NEXT: s_add_i32 s5, s5, 3
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s79, 16
+; SI-NEXT: s_add_i32 s40, s4, 0x30000
+; SI-NEXT: s_add_i32 s4, s91, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s77, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: s_and_b32 s6, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s37, 16
+; SI-NEXT: s_add_i32 s41, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s37, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_and_b32 s7, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s77, 16
+; SI-NEXT: s_add_i32 s42, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s75, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s8, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s36, 16
+; SI-NEXT: s_add_i32 s43, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s36, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: s_and_b32 s9, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s10, s75, 16
+; SI-NEXT: s_add_i32 s44, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s73, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s10, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s35, 16
+; SI-NEXT: s_add_i32 s45, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s35, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: s_and_b32 s11, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s12, s73, 16
+; SI-NEXT: s_add_i32 s46, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s63, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_and_b32 s12, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s34, 16
+; SI-NEXT: s_add_i32 s47, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s34, 16
; SI-NEXT: s_add_i32 s23, s23, 3
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s13, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s14, vcc_hi, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_and_b32 s5, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: s_and_b32 s14, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s31, 16
+; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s7, s31, 16
; SI-NEXT: s_add_i32 s21, s21, 3
-; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: s_and_b32 s15, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s20, vcc_lo, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
+; SI-NEXT: s_and_b32 s7, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s8, vcc_lo, 16
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s15, s20, s15
-; SI-NEXT: s_and_b32 s18, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s30, 16
-; SI-NEXT: s_or_b32 s18, s20, s18
+; SI-NEXT: s_or_b32 s7, s8, s7
+; SI-NEXT: s_and_b32 s8, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s9, s30, 16
; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_add_i32 s40, s18, 0x30000
-; SI-NEXT: s_and_b32 s18, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s95, 16
-; SI-NEXT: s_or_b32 s18, s19, s18
+; SI-NEXT: s_or_b32 s8, s9, s8
+; SI-NEXT: s_and_b32 s9, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_add_i32 s41, s18, 0x30000
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s57, 16
-; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s9, s10, s9
+; SI-NEXT: s_and_b32 s10, s16, 0xffff
+; SI-NEXT: s_lshl_b32 s11, s57, 16
; SI-NEXT: s_add_i32 s17, s17, 3
-; SI-NEXT: s_add_i32 s42, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s94, 16
+; SI-NEXT: s_or_b32 s10, s11, s10
+; SI-NEXT: s_and_b32 s11, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s94, 16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
+; SI-NEXT: s_or_b32 s11, s16, s11
; SI-NEXT: s_add_i32 s6, s6, 0x30000
; SI-NEXT: s_add_i32 s7, s7, 0x30000
; SI-NEXT: s_add_i32 s8, s8, 0x30000
; SI-NEXT: s_add_i32 s9, s9, 0x30000
; SI-NEXT: s_add_i32 s10, s10, 0x30000
; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s12, s12, 0x30000
-; SI-NEXT: s_add_i32 s13, s13, 0x30000
-; SI-NEXT: s_add_i32 s14, s14, 0x30000
-; SI-NEXT: s_add_i32 s15, s15, 0x30000
-; SI-NEXT: s_add_i32 s43, s16, 0x30000
-; SI-NEXT: s_lshr_b64 s[72:73], s[10:11], 16
-; SI-NEXT: s_lshr_b64 s[74:75], s[8:9], 16
-; SI-NEXT: s_lshr_b64 s[76:77], s[6:7], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[4:5], 16
-; SI-NEXT: s_lshr_b64 s[88:89], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[90:91], s[44:45], 16
-; SI-NEXT: s_lshr_b64 s[56:57], s[42:43], 16
-; SI-NEXT: s_lshr_b64 s[58:59], s[40:41], 16
-; SI-NEXT: s_lshr_b64 s[60:61], s[14:15], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[12:13], 16
-; SI-NEXT: s_lshr_b32 s94, s43, 16
-; SI-NEXT: s_lshr_b32 s95, s41, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, s15, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s13, 16
-; SI-NEXT: s_lshr_b32 s73, s11, 16
-; SI-NEXT: s_lshr_b32 s75, s9, 16
-; SI-NEXT: s_lshr_b32 s77, s7, 16
-; SI-NEXT: s_lshr_b32 s79, s5, 16
-; SI-NEXT: s_lshr_b32 s89, s47, 16
-; SI-NEXT: s_lshr_b32 s91, s45, 16
+; SI-NEXT: s_lshr_b64 s[62:63], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[72:73], s[46:47], 16
+; SI-NEXT: s_lshr_b64 s[74:75], s[44:45], 16
+; SI-NEXT: s_lshr_b64 s[76:77], s[42:43], 16
+; SI-NEXT: s_lshr_b64 s[78:79], s[40:41], 16
+; SI-NEXT: s_lshr_b64 s[88:89], s[14:15], 16
+; SI-NEXT: s_lshr_b64 s[56:57], s[10:11], 16
+; SI-NEXT: s_lshr_b64 s[58:59], s[8:9], 16
+; SI-NEXT: s_lshr_b64 s[60:61], s[6:7], 16
+; SI-NEXT: s_lshr_b64 s[90:91], s[12:13], 16
+; SI-NEXT: s_lshr_b32 s94, s11, 16
+; SI-NEXT: s_lshr_b32 s95, s9, 16
+; SI-NEXT: s_lshr_b32 vcc_lo, s7, 16
+; SI-NEXT: s_lshr_b32 vcc_hi, s5, 16
+; SI-NEXT: s_lshr_b32 s63, s47, 16
+; SI-NEXT: s_lshr_b32 s73, s45, 16
+; SI-NEXT: s_lshr_b32 s75, s43, 16
+; SI-NEXT: s_lshr_b32 s77, s41, 16
+; SI-NEXT: s_lshr_b32 s79, s15, 16
+; SI-NEXT: s_lshr_b32 s89, s13, 16
; SI-NEXT: .LBB57_5: ; %end
-; SI-NEXT: s_and_b32 s16, s42, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s56, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_and_b32 s17, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s94, 16
-; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_and_b32 s18, s40, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s58, 16
-; SI-NEXT: s_or_b32 s18, s18, s19
-; SI-NEXT: s_and_b32 s19, s41, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s95, 16
-; SI-NEXT: s_or_b32 s19, s19, s20
-; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s60, 16
-; SI-NEXT: s_or_b32 s14, s14, s20
-; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s20, vcc_lo, 16
-; SI-NEXT: s_or_b32 s15, s15, s20
-; SI-NEXT: s_and_b32 s12, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s62, 16
-; SI-NEXT: s_or_b32 s12, s12, s20
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s20, vcc_hi, 16
-; SI-NEXT: s_or_b32 s13, s13, s20
; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s72, 16
-; SI-NEXT: s_or_b32 s10, s10, s20
+; SI-NEXT: s_lshl_b32 s16, s56, 16
+; SI-NEXT: s_or_b32 s10, s10, s16
; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s73, 16
-; SI-NEXT: s_or_b32 s11, s11, s20
+; SI-NEXT: s_lshl_b32 s16, s94, 16
+; SI-NEXT: s_or_b32 s11, s11, s16
; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s74, 16
-; SI-NEXT: s_or_b32 s8, s8, s20
+; SI-NEXT: s_lshl_b32 s16, s58, 16
+; SI-NEXT: s_or_b32 s8, s8, s16
; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s75, 16
-; SI-NEXT: s_or_b32 s9, s9, s20
+; SI-NEXT: s_lshl_b32 s16, s95, 16
+; SI-NEXT: s_or_b32 s9, s9, s16
; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s76, 16
-; SI-NEXT: s_or_b32 s6, s6, s20
+; SI-NEXT: s_lshl_b32 s16, s60, 16
+; SI-NEXT: s_or_b32 s6, s6, s16
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s77, 16
-; SI-NEXT: s_or_b32 s7, s7, s20
+; SI-NEXT: s_lshl_b32 s16, vcc_lo, 16
+; SI-NEXT: s_or_b32 s7, s7, s16
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s78, 16
-; SI-NEXT: s_or_b32 s4, s4, s20
+; SI-NEXT: s_lshl_b32 s16, s62, 16
+; SI-NEXT: s_or_b32 s4, s4, s16
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s79, 16
-; SI-NEXT: s_or_b32 s5, s5, s20
-; SI-NEXT: s_and_b32 s20, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s88, 16
+; SI-NEXT: s_lshl_b32 s16, vcc_hi, 16
+; SI-NEXT: s_or_b32 s5, s5, s16
+; SI-NEXT: s_and_b32 s16, s46, 0xffff
+; SI-NEXT: s_lshl_b32 s17, s72, 16
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_and_b32 s17, s47, 0xffff
+; SI-NEXT: s_lshl_b32 s18, s63, 16
+; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_and_b32 s18, s44, 0xffff
+; SI-NEXT: s_lshl_b32 s19, s74, 16
+; SI-NEXT: s_or_b32 s18, s18, s19
+; SI-NEXT: s_and_b32 s19, s45, 0xffff
+; SI-NEXT: s_lshl_b32 s20, s73, 16
+; SI-NEXT: s_or_b32 s19, s19, s20
+; SI-NEXT: s_and_b32 s20, s42, 0xffff
+; SI-NEXT: s_lshl_b32 s21, s76, 16
; SI-NEXT: s_or_b32 s20, s20, s21
-; SI-NEXT: s_and_b32 s21, s47, 0xffff
-; SI-NEXT: s_lshl_b32 s22, s89, 16
+; SI-NEXT: s_and_b32 s21, s43, 0xffff
+; SI-NEXT: s_lshl_b32 s22, s75, 16
; SI-NEXT: s_or_b32 s21, s21, s22
-; SI-NEXT: s_and_b32 s22, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s90, 16
+; SI-NEXT: s_and_b32 s22, s40, 0xffff
+; SI-NEXT: s_lshl_b32 s23, s78, 16
; SI-NEXT: s_or_b32 s22, s22, s23
-; SI-NEXT: s_and_b32 s23, s45, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s91, 16
+; SI-NEXT: s_and_b32 s23, s41, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s77, 16
; SI-NEXT: s_or_b32 s23, s23, s24
+; SI-NEXT: s_and_b32 s14, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s88, 16
+; SI-NEXT: s_or_b32 s14, s14, s24
+; SI-NEXT: s_and_b32 s15, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s79, 16
+; SI-NEXT: s_or_b32 s15, s15, s24
+; SI-NEXT: s_and_b32 s12, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s90, 16
+; SI-NEXT: s_or_b32 s12, s12, s24
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s89, 16
+; SI-NEXT: s_or_b32 s13, s13, s24
; SI-NEXT: v_readlane_b32 s30, v20, 8
-; SI-NEXT: v_mov_b32_e32 v0, s16
-; SI-NEXT: v_mov_b32_e32 v1, s17
-; SI-NEXT: v_mov_b32_e32 v2, s18
-; SI-NEXT: v_mov_b32_e32 v3, s19
-; SI-NEXT: v_mov_b32_e32 v4, s14
-; SI-NEXT: v_mov_b32_e32 v5, s15
-; SI-NEXT: v_mov_b32_e32 v6, s12
-; SI-NEXT: v_mov_b32_e32 v7, s13
-; SI-NEXT: v_mov_b32_e32 v8, s10
-; SI-NEXT: v_mov_b32_e32 v9, s11
-; SI-NEXT: v_mov_b32_e32 v10, s8
-; SI-NEXT: v_mov_b32_e32 v11, s9
-; SI-NEXT: v_mov_b32_e32 v12, s6
-; SI-NEXT: v_mov_b32_e32 v13, s7
-; SI-NEXT: v_mov_b32_e32 v14, s4
-; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_mov_b32_e32 v16, s20
-; SI-NEXT: v_mov_b32_e32 v17, s21
-; SI-NEXT: v_mov_b32_e32 v18, s22
-; SI-NEXT: v_mov_b32_e32 v19, s23
+; SI-NEXT: v_mov_b32_e32 v0, s10
+; SI-NEXT: v_mov_b32_e32 v1, s11
+; SI-NEXT: v_mov_b32_e32 v2, s8
+; SI-NEXT: v_mov_b32_e32 v3, s9
+; SI-NEXT: v_mov_b32_e32 v4, s6
+; SI-NEXT: v_mov_b32_e32 v5, s7
+; SI-NEXT: v_mov_b32_e32 v6, s4
+; SI-NEXT: v_mov_b32_e32 v7, s5
+; SI-NEXT: v_mov_b32_e32 v8, s16
+; SI-NEXT: v_mov_b32_e32 v9, s17
+; SI-NEXT: v_mov_b32_e32 v10, s18
+; SI-NEXT: v_mov_b32_e32 v11, s19
+; SI-NEXT: v_mov_b32_e32 v12, s20
+; SI-NEXT: v_mov_b32_e32 v13, s21
+; SI-NEXT: v_mov_b32_e32 v14, s22
+; SI-NEXT: v_mov_b32_e32 v15, s23
+; SI-NEXT: v_mov_b32_e32 v16, s14
+; SI-NEXT: v_mov_b32_e32 v17, s15
+; SI-NEXT: v_mov_b32_e32 v18, s12
+; SI-NEXT: v_mov_b32_e32 v19, s13
; SI-NEXT: v_readlane_b32 s31, v20, 9
; SI-NEXT: v_readlane_b32 s49, v20, 7
; SI-NEXT: v_readlane_b32 s48, v20, 6
@@ -33233,6 +33233,7 @@ define <40 x i16> @bitcast_v40f16_to_v40i16(<40 x half> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v18, v38, v18, s6
; GFX9-NEXT: v_perm_b32 v17, v37, v17, s6
; GFX9-NEXT: v_perm_b32 v16, v36, v16, s6
+; GFX9-NEXT: s_movk_i32 s7, 0x200
; GFX9-NEXT: v_perm_b32 v15, v35, v15, s6
; GFX9-NEXT: v_perm_b32 v14, v34, v14, s6
; GFX9-NEXT: v_perm_b32 v13, v33, v13, s6
@@ -33240,7 +33241,6 @@ define <40 x i16> @bitcast_v40f16_to_v40i16(<40 x half> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v11, v31, v11, s6
; GFX9-NEXT: v_perm_b32 v10, v30, v10, s6
; GFX9-NEXT: v_perm_b32 v9, v29, v9, s6
-; GFX9-NEXT: s_movk_i32 s7, 0x200
; GFX9-NEXT: v_perm_b32 v8, v28, v8, s6
; GFX9-NEXT: v_perm_b32 v7, v27, v7, s6
; GFX9-NEXT: v_perm_b32 v6, v26, v6, s6
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
index 048bfa8132a1d..d14f05df66742 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
@@ -34833,322 +34833,316 @@ define inreg <44 x half> @bitcast_v44i16_to_v44f16_scalar(<44 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v22, s35, 1
; SI-NEXT: v_writelane_b32 v22, s36, 2
; SI-NEXT: v_writelane_b32 v22, s37, 3
-; SI-NEXT: v_writelane_b32 v22, s38, 4
-; SI-NEXT: v_writelane_b32 v22, s39, 5
-; SI-NEXT: v_writelane_b32 v22, s48, 6
-; SI-NEXT: v_writelane_b32 v22, s49, 7
-; SI-NEXT: v_writelane_b32 v22, s50, 8
-; SI-NEXT: v_writelane_b32 v22, s51, 9
-; SI-NEXT: v_writelane_b32 v22, s52, 10
-; SI-NEXT: v_writelane_b32 v22, s53, 11
-; SI-NEXT: v_writelane_b32 v22, s54, 12
-; SI-NEXT: v_writelane_b32 v22, s55, 13
-; SI-NEXT: v_writelane_b32 v22, s64, 14
-; SI-NEXT: v_writelane_b32 v22, s65, 15
-; SI-NEXT: v_writelane_b32 v22, s66, 16
-; SI-NEXT: v_writelane_b32 v22, s67, 17
-; SI-NEXT: v_writelane_b32 v22, s68, 18
-; SI-NEXT: v_writelane_b32 v22, s69, 19
-; SI-NEXT: v_writelane_b32 v22, s70, 20
-; SI-NEXT: v_writelane_b32 v22, s30, 21
-; SI-NEXT: v_writelane_b32 v22, s31, 22
-; SI-NEXT: v_readfirstlane_b32 s15, v1
+; SI-NEXT: v_writelane_b32 v22, s30, 4
+; SI-NEXT: v_writelane_b32 v22, s31, 5
+; SI-NEXT: s_lshr_b32 s12, s16, 16
; SI-NEXT: ; implicit-def: $vgpr23 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s5, v7
-; SI-NEXT: v_readfirstlane_b32 s66, v6
-; SI-NEXT: v_readfirstlane_b32 s7, v5
-; SI-NEXT: v_readfirstlane_b32 s64, v4
-; SI-NEXT: v_readfirstlane_b32 s9, v3
-; SI-NEXT: v_readfirstlane_b32 s54, v2
-; SI-NEXT: v_readfirstlane_b32 s52, v0
+; SI-NEXT: s_lshr_b32 s10, s18, 16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v23, s15, 0
-; SI-NEXT: s_lshr_b32 s91, s29, 16
-; SI-NEXT: s_lshr_b32 s51, s28, 16
-; SI-NEXT: s_lshr_b32 s89, s27, 16
-; SI-NEXT: s_lshr_b32 s50, s26, 16
-; SI-NEXT: s_lshr_b32 s36, s25, 16
-; SI-NEXT: s_lshr_b32 s49, s24, 16
-; SI-NEXT: s_lshr_b32 s35, s23, 16
-; SI-NEXT: s_lshr_b32 s48, s22, 16
-; SI-NEXT: s_lshr_b32 s34, s21, 16
-; SI-NEXT: s_lshr_b32 s39, s20, 16
+; SI-NEXT: v_writelane_b32 v23, s12, 0
+; SI-NEXT: s_lshr_b32 s8, s20, 16
+; SI-NEXT: v_writelane_b32 v23, s10, 1
+; SI-NEXT: s_lshr_b32 s6, s22, 16
+; SI-NEXT: v_writelane_b32 v23, s8, 2
+; SI-NEXT: s_lshr_b32 s4, s24, 16
+; SI-NEXT: v_writelane_b32 v23, s6, 3
+; SI-NEXT: s_lshr_b32 s15, s26, 16
+; SI-NEXT: v_writelane_b32 v23, s4, 4
+; SI-NEXT: s_lshr_b32 s14, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v0
+; SI-NEXT: v_writelane_b32 v23, s15, 5
+; SI-NEXT: v_readfirstlane_b32 s31, v2
+; SI-NEXT: s_lshr_b32 s42, s30, 16
+; SI-NEXT: v_writelane_b32 v23, s14, 6
+; SI-NEXT: v_readfirstlane_b32 s34, v4
+; SI-NEXT: s_lshr_b32 s40, s31, 16
+; SI-NEXT: v_writelane_b32 v23, s42, 7
+; SI-NEXT: s_lshr_b32 s41, s34, 16
+; SI-NEXT: v_writelane_b32 v23, s40, 8
+; SI-NEXT: v_readfirstlane_b32 s13, v1
+; SI-NEXT: v_writelane_b32 v23, s41, 9
+; SI-NEXT: v_readfirstlane_b32 s11, v3
+; SI-NEXT: v_writelane_b32 v23, s13, 10
+; SI-NEXT: v_readfirstlane_b32 s7, v7
+; SI-NEXT: v_readfirstlane_b32 s35, v6
+; SI-NEXT: v_readfirstlane_b32 s9, v5
+; SI-NEXT: v_writelane_b32 v23, s11, 11
+; SI-NEXT: s_lshr_b32 s79, s29, 16
+; SI-NEXT: s_lshr_b32 s77, s27, 16
+; SI-NEXT: s_lshr_b32 s75, s25, 16
+; SI-NEXT: s_lshr_b32 s73, s23, 16
+; SI-NEXT: s_lshr_b32 s63, s21, 16
; SI-NEXT: s_lshr_b32 s61, s19, 16
-; SI-NEXT: s_lshr_b32 s38, s18, 16
-; SI-NEXT: s_lshr_b32 s70, s17, 16
-; SI-NEXT: s_lshr_b32 s63, s16, 16
-; SI-NEXT: s_lshr_b32 s37, s5, 16
-; SI-NEXT: s_lshr_b32 s65, s66, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s55, s64, 16
-; SI-NEXT: s_lshr_b32 s95, s9, 16
-; SI-NEXT: s_lshr_b32 s53, s54, 16
-; SI-NEXT: s_lshr_b32 s93, s15, 16
-; SI-NEXT: s_lshr_b32 s73, s52, 16
-; SI-NEXT: v_readfirstlane_b32 s4, v8
-; SI-NEXT: v_writelane_b32 v23, s9, 1
-; SI-NEXT: s_cmp_lg_u32 s4, 0
-; SI-NEXT: v_writelane_b32 v23, s7, 2
-; SI-NEXT: v_writelane_b32 v23, s5, 3
+; SI-NEXT: s_lshr_b32 s59, s17, 16
+; SI-NEXT: s_lshr_b32 s95, s7, 16
+; SI-NEXT: s_lshr_b32 s5, s35, 16
+; SI-NEXT: s_lshr_b32 s93, s9, 16
+; SI-NEXT: s_lshr_b32 s91, s11, 16
+; SI-NEXT: s_lshr_b32 s89, s13, 16
+; SI-NEXT: v_readfirstlane_b32 s43, v8
+; SI-NEXT: v_writelane_b32 v23, s9, 12
+; SI-NEXT: s_cmp_lg_u32 s43, 0
+; SI-NEXT: v_writelane_b32 v23, s7, 13
+; SI-NEXT: v_writelane_b32 v23, s5, 14
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s13, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s70, 16
-; SI-NEXT: s_lshl_b32 s62, s63, 16
-; SI-NEXT: s_mov_b32 s11, s63
-; SI-NEXT: s_or_b32 s63, s13, s41
-; SI-NEXT: s_and_b32 s41, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s61, 16
-; SI-NEXT: s_lshl_b32 s92, s73, 16
-; SI-NEXT: s_mov_b32 s13, s73
-; SI-NEXT: s_or_b32 s73, s41, s43
-; SI-NEXT: s_and_b32 s41, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s34, 16
-; SI-NEXT: s_or_b32 s75, s41, s43
-; SI-NEXT: s_and_b32 s41, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s35, 16
-; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_or_b32 s77, s41, s43
-; SI-NEXT: s_and_b32 s41, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s36, 16
-; SI-NEXT: s_or_b32 s56, s4, s62
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s72, s38, 16
-; SI-NEXT: s_lshr_b64 s[58:59], s[62:63], 16
-; SI-NEXT: s_or_b32 s79, s41, s43
-; SI-NEXT: s_and_b32 s41, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s72
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s74, s39, 16
-; SI-NEXT: v_writelane_b32 v23, s58, 4
-; SI-NEXT: s_mov_b32 s60, s89
-; SI-NEXT: s_or_b32 s89, s41, s43
-; SI-NEXT: s_and_b32 s41, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s74
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s76, s48, 16
-; SI-NEXT: v_writelane_b32 v23, s59, 5
-; SI-NEXT: s_mov_b32 s58, s91
-; SI-NEXT: s_or_b32 s91, s41, s43
-; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s76
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s78, s49, 16
-; SI-NEXT: s_mov_b32 s62, s93
-; SI-NEXT: s_or_b32 s93, s15, s41
+; SI-NEXT: s_lshl_b32 s46, s15, 16
+; SI-NEXT: s_lshl_b32 s44, s14, 16
+; SI-NEXT: s_lshl_b32 s14, s41, 16
+; SI-NEXT: s_and_b32 s15, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s59, 16
+; SI-NEXT: s_mov_b32 s74, s59
+; SI-NEXT: s_or_b32 s59, s15, s41
+; SI-NEXT: s_and_b32 s15, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s61, 16
+; SI-NEXT: s_mov_b32 s76, s61
+; SI-NEXT: s_or_b32 s61, s15, s41
+; SI-NEXT: s_and_b32 s15, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s63, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s12, 16
+; SI-NEXT: s_mov_b32 s78, s63
+; SI-NEXT: s_or_b32 s63, s15, s41
+; SI-NEXT: s_and_b32 s15, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s73, 16
+; SI-NEXT: s_or_b32 s12, s43, s58
+; SI-NEXT: s_and_b32 s43, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s10, 16
+; SI-NEXT: s_mov_b32 s88, s73
+; SI-NEXT: s_or_b32 s73, s15, s41
+; SI-NEXT: s_and_b32 s15, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s75, 16
+; SI-NEXT: s_or_b32 s10, s43, s60
+; SI-NEXT: s_and_b32 s43, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s8, 16
+; SI-NEXT: s_or_b32 s57, s15, s41
+; SI-NEXT: s_and_b32 s15, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s77, 16
+; SI-NEXT: s_or_b32 s8, s43, s62
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s72, s6, 16
+; SI-NEXT: s_or_b32 s47, s15, s41
+; SI-NEXT: s_and_b32 s15, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s79, 16
+; SI-NEXT: s_or_b32 s6, s43, s72
+; SI-NEXT: s_and_b32 s43, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s4, 16
+; SI-NEXT: s_or_b32 s45, s15, s41
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s15, s89, 16
+; SI-NEXT: s_or_b32 s4, s43, s56
+; SI-NEXT: s_or_b32 s43, s13, s15
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s91, 16
+; SI-NEXT: s_or_b32 s41, s11, s13
; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s78
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s88, s50, 16
-; SI-NEXT: s_mov_b32 s59, s95
-; SI-NEXT: s_or_b32 s95, s9, s15
+; SI-NEXT: s_lshl_b32 s11, s93, 16
+; SI-NEXT: s_or_b32 s15, s9, s11
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s67, 16
-; SI-NEXT: s_or_b32 s14, s4, s88
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s90, s51, 16
-; SI-NEXT: s_or_b32 s31, s7, s9
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s37, 16
-; SI-NEXT: s_or_b32 s12, s4, s90
-; SI-NEXT: s_and_b32 s4, s52, 0xffff
-; SI-NEXT: s_or_b32 vcc_hi, s5, s7
-; SI-NEXT: s_mov_b32 s5, s34
-; SI-NEXT: s_mov_b32 s7, s35
-; SI-NEXT: s_lshr_b64 s[34:35], s[72:73], 16
-; SI-NEXT: s_or_b32 s10, s4, s92
-; SI-NEXT: s_and_b32 s4, s54, 0xffff
-; SI-NEXT: s_lshl_b32 s94, s53, 16
-; SI-NEXT: v_writelane_b32 v23, s34, 6
-; SI-NEXT: s_or_b32 s8, s4, s94
-; SI-NEXT: s_and_b32 s4, s64, 0xffff
-; SI-NEXT: s_lshl_b32 s30, s55, 16
-; SI-NEXT: s_lshl_b32 vcc_lo, s65, 16
-; SI-NEXT: v_writelane_b32 v23, s35, 7
-; SI-NEXT: s_lshr_b64 s[68:69], s[74:75], 16
-; SI-NEXT: s_mov_b32 s47, s73
-; SI-NEXT: s_lshr_b64 s[72:73], s[76:77], 16
-; SI-NEXT: s_or_b32 s6, s4, s30
-; SI-NEXT: s_and_b32 s4, s66, 0xffff
-; SI-NEXT: s_mov_b32 s57, s63
-; SI-NEXT: v_writelane_b32 v23, s68, 8
-; SI-NEXT: s_mov_b32 s63, s11
-; SI-NEXT: s_mov_b32 s73, s13
-; SI-NEXT: s_mov_b32 s45, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[78:79], 16
-; SI-NEXT: s_mov_b32 s43, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[88:89], 16
-; SI-NEXT: s_mov_b32 s41, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[90:91], 16
-; SI-NEXT: s_mov_b32 s15, s89
-; SI-NEXT: s_lshr_b64 s[88:89], s[92:93], 16
-; SI-NEXT: s_mov_b32 s13, s91
-; SI-NEXT: s_lshr_b64 s[90:91], s[94:95], 16
-; SI-NEXT: s_mov_b32 s11, s93
-; SI-NEXT: s_lshr_b64 s[92:93], s[30:31], 16
-; SI-NEXT: s_mov_b32 s9, s95
-; SI-NEXT: s_lshr_b64 s[94:95], vcc, 16
-; SI-NEXT: s_or_b32 s4, s4, vcc_lo
-; SI-NEXT: s_mov_b32 s35, s7
-; SI-NEXT: s_mov_b32 s34, s5
-; SI-NEXT: v_writelane_b32 v23, s69, 9
-; SI-NEXT: s_mov_b32 s89, s60
-; SI-NEXT: s_mov_b32 s91, s58
-; SI-NEXT: s_mov_b32 s93, s62
-; SI-NEXT: s_mov_b32 s95, s59
-; SI-NEXT: s_mov_b32 s7, s31
-; SI-NEXT: s_mov_b64 s[30:31], 0
-; SI-NEXT: s_mov_b32 s5, vcc_hi
+; SI-NEXT: s_lshl_b32 s9, s95, 16
+; SI-NEXT: s_mov_b32 s13, s59
+; SI-NEXT: s_lshr_b64 s[58:59], s[58:59], 16
+; SI-NEXT: s_mov_b32 s11, s61
+; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
+; SI-NEXT: s_or_b32 s37, s7, s9
+; SI-NEXT: s_mov_b32 s59, s74
+; SI-NEXT: s_mov_b32 s61, s76
+; SI-NEXT: s_mov_b32 s9, s63
+; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
+; SI-NEXT: s_mov_b32 s76, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[56:57], 16
+; SI-NEXT: s_mov_b32 s63, s78
+; SI-NEXT: s_mov_b32 s7, s73
+; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
+; SI-NEXT: s_mov_b32 s75, s76
+; SI-NEXT: s_and_b32 s56, s26, 0xffff
+; SI-NEXT: s_mov_b32 s78, s77
+; SI-NEXT: s_lshr_b64 s[76:77], s[46:47], 16
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_mov_b32 s73, s88
+; SI-NEXT: s_or_b32 s56, s56, s46
+; SI-NEXT: s_mov_b32 s77, s78
+; SI-NEXT: s_and_b32 s46, s28, 0xffff
+; SI-NEXT: s_mov_b32 s88, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[44:45], 16
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s46, s46, s44
+; SI-NEXT: s_mov_b32 s79, s88
+; SI-NEXT: s_and_b32 s44, s30, 0xffff
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_lshr_b64 s[88:89], s[42:43], 16
+; SI-NEXT: s_or_b32 s44, s44, s42
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_and_b32 s42, s31, 0xffff
+; SI-NEXT: s_mov_b32 s92, s91
+; SI-NEXT: s_lshr_b64 s[90:91], s[40:41], 16
+; SI-NEXT: s_lshl_b32 s36, s5, 16
+; SI-NEXT: s_or_b32 s42, s42, s40
+; SI-NEXT: s_mov_b32 s91, s92
+; SI-NEXT: s_and_b32 s40, s34, 0xffff
+; SI-NEXT: s_mov_b32 s94, s93
+; SI-NEXT: s_lshr_b64 s[92:93], s[14:15], 16
+; SI-NEXT: s_or_b32 s40, s40, s14
+; SI-NEXT: s_mov_b32 s93, s94
+; SI-NEXT: s_and_b32 s14, s35, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_lshr_b64 s[94:95], s[36:37], 16
+; SI-NEXT: s_mov_b32 s5, s57
+; SI-NEXT: s_mov_b32 s57, s47
+; SI-NEXT: s_mov_b32 s47, s45
+; SI-NEXT: s_mov_b32 s45, s43
+; SI-NEXT: s_mov_b32 s43, s41
+; SI-NEXT: s_mov_b32 s41, s15
+; SI-NEXT: s_or_b32 s14, s14, s36
+; SI-NEXT: s_mov_b32 s15, s37
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b64 vcc, 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: s_mov_b64 s[30:31], -1
-; SI-NEXT: v_writelane_b32 v23, s4, 4
-; SI-NEXT: v_writelane_b32 v23, s5, 5
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr42
+; SI-NEXT: s_mov_b64 vcc, -1
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr58
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr60
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr56
; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr46
; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr44
; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr90
-; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr92
+; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr94
-; SI-NEXT: v_writelane_b32 v23, s4, 6
-; SI-NEXT: v_writelane_b32 v23, s5, 7
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: v_writelane_b32 v23, s4, 8
-; SI-NEXT: v_writelane_b32 v23, s5, 9
-; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB57_3: ; %Flow
-; SI-NEXT: s_and_b64 s[58:59], s[30:31], exec
-; SI-NEXT: v_readlane_b32 s30, v23, 4
-; SI-NEXT: v_readlane_b32 s31, v23, 5
-; SI-NEXT: s_mov_b32 vcc_lo, s30
-; SI-NEXT: v_readlane_b32 s30, v23, 6
-; SI-NEXT: v_readlane_b32 s31, v23, 7
-; SI-NEXT: s_cselect_b32 s58, 1, 0
-; SI-NEXT: s_mov_b32 s60, s30
-; SI-NEXT: v_readlane_b32 s30, v23, 8
-; SI-NEXT: s_cmp_lg_u32 s58, 1
-; SI-NEXT: s_mov_b32 s58, s30
-; SI-NEXT: v_readlane_b32 s31, v23, 9
+; SI-NEXT: s_and_b64 vcc, vcc, exec
+; SI-NEXT: s_cselect_b32 vcc_lo, 1, 0
+; SI-NEXT: s_cmp_lg_u32 vcc_lo, 1
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s4, s66, 3
+; SI-NEXT: s_add_i32 s35, s35, 3
+; SI-NEXT: v_readlane_b32 s4, v23, 14
+; SI-NEXT: s_and_b32 s5, s35, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: s_add_i32 s14, s4, 0x30000
+; SI-NEXT: v_readlane_b32 s4, v23, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s65, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v23, 3
-; SI-NEXT: s_add_i32 s5, s5, 3
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s37, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: s_add_i32 s6, s64, 3
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s55, 16
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v23, 2
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s67, 16
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_add_i32 s8, s54, 3
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s53, 16
-; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v23, 1
-; SI-NEXT: s_add_i32 s9, s9, 3
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s10, s95, 16
-; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_add_i32 s10, s52, 3
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s73, 16
-; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v23, 0
-; SI-NEXT: s_add_i32 s11, s11, 3
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s12, s93, 16
+; SI-NEXT: s_add_i32 s15, s4, 0x30000
+; SI-NEXT: s_add_i32 s34, s34, 3
+; SI-NEXT: v_readlane_b32 s4, v23, 9
+; SI-NEXT: s_and_b32 s5, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: s_add_i32 s40, s4, 0x30000
+; SI-NEXT: v_readlane_b32 s4, v23, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s93, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s41, s4, 0x30000
+; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v23, 8
+; SI-NEXT: s_and_b32 s5, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: s_add_i32 s42, s4, 0x30000
+; SI-NEXT: v_readlane_b32 s4, v23, 11
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s91, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s43, s4, 0x30000
+; SI-NEXT: s_add_i32 s30, s30, 3
+; SI-NEXT: v_readlane_b32 s4, v23, 7
+; SI-NEXT: s_and_b32 s5, s30, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: s_add_i32 s44, s4, 0x30000
+; SI-NEXT: v_readlane_b32 s4, v23, 10
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s89, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_and_b32 s12, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s51, 16
+; SI-NEXT: v_readlane_b32 s4, v23, 6
+; SI-NEXT: s_and_b32 s5, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s13, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s14, s91, 16
+; SI-NEXT: s_add_i32 s46, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s79, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: s_and_b32 s14, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s50, 16
+; SI-NEXT: v_readlane_b32 s4, v23, 5
+; SI-NEXT: s_and_b32 s5, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: s_and_b32 s15, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s89, 16
+; SI-NEXT: s_add_i32 s56, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s77, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s57, s4, 0x30000
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_or_b32 s15, s26, s15
-; SI-NEXT: s_and_b32 s24, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s49, 16
-; SI-NEXT: s_or_b32 s24, s26, s24
+; SI-NEXT: v_readlane_b32 s4, v23, 4
+; SI-NEXT: s_and_b32 s5, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_add_i32 s40, s24, 0x30000
-; SI-NEXT: s_and_b32 s24, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s25, s36, 16
-; SI-NEXT: s_or_b32 s24, s25, s24
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: s_and_b32 s5, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s75, 16
+; SI-NEXT: s_or_b32 s5, s6, s5
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_add_i32 s41, s24, 0x30000
-; SI-NEXT: s_and_b32 s22, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s48, 16
-; SI-NEXT: s_or_b32 s22, s24, s22
+; SI-NEXT: v_readlane_b32 s6, v23, 3
+; SI-NEXT: s_and_b32 s7, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_add_i32 s23, s23, 3
-; SI-NEXT: s_add_i32 s42, s22, 0x30000
-; SI-NEXT: s_and_b32 s22, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s35, 16
-; SI-NEXT: s_or_b32 s22, s23, s22
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_and_b32 s7, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s8, s73, 16
+; SI-NEXT: s_or_b32 s7, s8, s7
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s43, s22, 0x30000
-; SI-NEXT: s_and_b32 s20, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s22, s39, 16
-; SI-NEXT: s_or_b32 s20, s22, s20
+; SI-NEXT: v_readlane_b32 s8, v23, 2
+; SI-NEXT: s_and_b32 s9, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_add_i32 s21, s21, 3
-; SI-NEXT: s_add_i32 s44, s20, 0x30000
-; SI-NEXT: s_and_b32 s20, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s34, 16
-; SI-NEXT: s_or_b32 s20, s21, s20
+; SI-NEXT: s_or_b32 s8, s8, s9
+; SI-NEXT: s_and_b32 s9, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s63, 16
+; SI-NEXT: s_or_b32 s9, s10, s9
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_add_i32 s45, s20, 0x30000
-; SI-NEXT: s_and_b32 s18, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s38, 16
-; SI-NEXT: s_or_b32 s18, s20, s18
+; SI-NEXT: v_readlane_b32 s10, v23, 1
+; SI-NEXT: s_and_b32 s11, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_add_i32 s46, s18, 0x30000
-; SI-NEXT: s_and_b32 s18, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s61, 16
-; SI-NEXT: s_or_b32 s18, s19, s18
+; SI-NEXT: s_or_b32 s10, s10, s11
+; SI-NEXT: s_and_b32 s11, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s61, 16
+; SI-NEXT: s_or_b32 s11, s12, s11
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_add_i32 s47, s18, 0x30000
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s63, 16
-; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: v_readlane_b32 s12, v23, 0
+; SI-NEXT: s_and_b32 s13, s16, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_add_i32 s17, s17, 3
-; SI-NEXT: s_add_i32 s56, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s70, 16
+; SI-NEXT: s_or_b32 s12, s12, s13
+; SI-NEXT: s_and_b32 s13, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s59, 16
+; SI-NEXT: s_or_b32 s13, s16, s13
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
@@ -35157,142 +35151,121 @@ define inreg <44 x half> @bitcast_v44i16_to_v44f16_scalar(<44 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s9, s9, 0x30000
; SI-NEXT: s_add_i32 s10, s10, 0x30000
; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s12, s12, 0x30000
; SI-NEXT: s_add_i32 s13, s13, 0x30000
-; SI-NEXT: s_add_i32 s14, s14, 0x30000
-; SI-NEXT: s_add_i32 s15, s15, 0x30000
-; SI-NEXT: s_add_i32 s57, s16, 0x30000
-; SI-NEXT: s_lshr_b64 s[60:61], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[88:89], s[10:11], 16
-; SI-NEXT: s_lshr_b64 s[90:91], s[8:9], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[6:7], 16
-; SI-NEXT: s_lshr_b64 s[94:95], s[4:5], 16
-; SI-NEXT: s_lshr_b64 vcc, s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[58:59], s[44:45], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[42:43], 16
-; SI-NEXT: s_lshr_b64 s[74:75], s[40:41], 16
-; SI-NEXT: s_lshr_b64 s[76:77], s[14:15], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[12:13], 16
-; SI-NEXT: s_lshr_b32 s70, s57, 16
-; SI-NEXT: s_lshr_b32 s61, s47, 16
-; SI-NEXT: s_lshr_b32 s34, s45, 16
-; SI-NEXT: s_lshr_b32 s35, s43, 16
-; SI-NEXT: s_lshr_b32 s36, s41, 16
-; SI-NEXT: s_lshr_b32 s89, s15, 16
-; SI-NEXT: s_lshr_b32 s91, s13, 16
-; SI-NEXT: s_lshr_b32 s93, s11, 16
-; SI-NEXT: s_lshr_b32 s95, s9, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s37, s5, 16
+; SI-NEXT: s_lshr_b64 s[58:59], s[12:13], 16
+; SI-NEXT: s_lshr_b64 s[60:61], s[10:11], 16
+; SI-NEXT: s_lshr_b64 s[62:63], s[8:9], 16
+; SI-NEXT: s_lshr_b64 s[72:73], s[6:7], 16
+; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[76:77], s[56:57], 16
+; SI-NEXT: s_lshr_b64 s[78:79], s[46:47], 16
+; SI-NEXT: s_lshr_b64 s[88:89], s[44:45], 16
+; SI-NEXT: s_lshr_b64 s[90:91], s[42:43], 16
+; SI-NEXT: s_lshr_b64 s[92:93], s[40:41], 16
+; SI-NEXT: s_lshr_b64 s[94:95], s[14:15], 16
+; SI-NEXT: s_lshr_b32 s59, s13, 16
+; SI-NEXT: s_lshr_b32 s61, s11, 16
+; SI-NEXT: s_lshr_b32 s63, s9, 16
+; SI-NEXT: s_lshr_b32 s73, s7, 16
+; SI-NEXT: s_lshr_b32 s75, s5, 16
+; SI-NEXT: s_lshr_b32 s77, s57, 16
+; SI-NEXT: s_lshr_b32 s79, s47, 16
+; SI-NEXT: s_lshr_b32 s89, s45, 16
+; SI-NEXT: s_lshr_b32 s91, s43, 16
+; SI-NEXT: s_lshr_b32 s93, s41, 16
+; SI-NEXT: s_lshr_b32 s95, s15, 16
; SI-NEXT: .LBB57_5: ; %end
+; SI-NEXT: s_and_b32 s12, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s58, 16
+; SI-NEXT: s_or_b32 s12, s12, s16
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s59, 16
+; SI-NEXT: s_or_b32 s13, s13, s16
+; SI-NEXT: s_and_b32 s10, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s60, 16
+; SI-NEXT: s_or_b32 s10, s10, s16
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s61, 16
+; SI-NEXT: s_or_b32 s11, s11, s16
+; SI-NEXT: s_and_b32 s8, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s62, 16
+; SI-NEXT: s_or_b32 s8, s8, s16
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s63, 16
+; SI-NEXT: s_or_b32 s9, s9, s16
+; SI-NEXT: s_and_b32 s6, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s6, s6, s16
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s73, 16
+; SI-NEXT: s_or_b32 s7, s7, s16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s74, 16
+; SI-NEXT: s_or_b32 s4, s4, s16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s16, s75, 16
+; SI-NEXT: s_or_b32 s5, s5, s16
; SI-NEXT: s_and_b32 s16, s56, 0xffff
-; SI-NEXT: s_lshl_b32 s17, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s17, s76, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s57, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s70, 16
+; SI-NEXT: s_lshl_b32 s18, s77, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s60, 16
+; SI-NEXT: s_lshl_b32 s19, s78, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s47, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s61, 16
+; SI-NEXT: s_lshl_b32 s20, s79, 16
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s20, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s58, 16
+; SI-NEXT: s_lshl_b32 s21, s88, 16
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_and_b32 s21, s45, 0xffff
-; SI-NEXT: s_lshl_b32 s22, s34, 16
+; SI-NEXT: s_lshl_b32 s22, s89, 16
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s22, s42, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s72, 16
+; SI-NEXT: s_lshl_b32 s23, s90, 16
; SI-NEXT: s_or_b32 s22, s22, s23
; SI-NEXT: s_and_b32 s23, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s35, 16
+; SI-NEXT: s_lshl_b32 s24, s91, 16
; SI-NEXT: s_or_b32 s23, s23, s24
; SI-NEXT: s_and_b32 s24, s40, 0xffff
-; SI-NEXT: s_lshl_b32 s25, s74, 16
+; SI-NEXT: s_lshl_b32 s25, s92, 16
; SI-NEXT: s_or_b32 s24, s24, s25
; SI-NEXT: s_and_b32 s25, s41, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s36, 16
+; SI-NEXT: s_lshl_b32 s26, s93, 16
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s76, 16
+; SI-NEXT: s_lshl_b32 s26, s94, 16
; SI-NEXT: s_or_b32 s14, s14, s26
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s89, 16
-; SI-NEXT: s_or_b32 s15, s15, s26
-; SI-NEXT: s_and_b32 s12, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s78, 16
-; SI-NEXT: s_or_b32 s12, s12, s26
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s91, 16
-; SI-NEXT: s_or_b32 s13, s13, s26
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s88, 16
-; SI-NEXT: s_or_b32 s10, s10, s26
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s93, 16
-; SI-NEXT: s_or_b32 s11, s11, s26
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s90, 16
-; SI-NEXT: s_or_b32 s8, s8, s26
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s26, s95, 16
-; SI-NEXT: s_or_b32 s9, s9, s26
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s92, 16
-; SI-NEXT: s_or_b32 s6, s6, s26
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s67, 16
-; SI-NEXT: s_or_b32 s7, s7, s26
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s94, 16
-; SI-NEXT: s_or_b32 s4, s4, s26
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s26, s37, 16
-; SI-NEXT: s_or_b32 s5, s5, s26
-; SI-NEXT: v_readlane_b32 s30, v22, 21
-; SI-NEXT: v_mov_b32_e32 v0, s16
-; SI-NEXT: v_mov_b32_e32 v1, s17
-; SI-NEXT: v_mov_b32_e32 v2, s18
-; SI-NEXT: v_mov_b32_e32 v3, s19
-; SI-NEXT: v_mov_b32_e32 v4, s20
-; SI-NEXT: v_mov_b32_e32 v5, s21
-; SI-NEXT: v_mov_b32_e32 v6, s22
-; SI-NEXT: v_mov_b32_e32 v7, s23
-; SI-NEXT: v_mov_b32_e32 v8, s24
-; SI-NEXT: v_mov_b32_e32 v9, s25
-; SI-NEXT: v_mov_b32_e32 v10, s14
-; SI-NEXT: v_mov_b32_e32 v11, s15
-; SI-NEXT: v_mov_b32_e32 v12, s12
-; SI-NEXT: v_mov_b32_e32 v13, s13
-; SI-NEXT: v_mov_b32_e32 v14, s10
-; SI-NEXT: v_mov_b32_e32 v15, s11
-; SI-NEXT: v_mov_b32_e32 v16, s8
-; SI-NEXT: v_mov_b32_e32 v17, s9
-; SI-NEXT: v_mov_b32_e32 v18, s6
-; SI-NEXT: v_mov_b32_e32 v19, s7
-; SI-NEXT: v_mov_b32_e32 v20, s4
-; SI-NEXT: v_mov_b32_e32 v21, s5
-; SI-NEXT: v_readlane_b32 s31, v22, 22
-; SI-NEXT: v_readlane_b32 s70, v22, 20
-; SI-NEXT: v_readlane_b32 s69, v22, 19
-; SI-NEXT: v_readlane_b32 s68, v22, 18
-; SI-NEXT: v_readlane_b32 s67, v22, 17
-; SI-NEXT: v_readlane_b32 s66, v22, 16
-; SI-NEXT: v_readlane_b32 s65, v22, 15
-; SI-NEXT: v_readlane_b32 s64, v22, 14
-; SI-NEXT: v_readlane_b32 s55, v22, 13
-; SI-NEXT: v_readlane_b32 s54, v22, 12
-; SI-NEXT: v_readlane_b32 s53, v22, 11
-; SI-NEXT: v_readlane_b32 s52, v22, 10
-; SI-NEXT: v_readlane_b32 s51, v22, 9
-; SI-NEXT: v_readlane_b32 s50, v22, 8
-; SI-NEXT: v_readlane_b32 s49, v22, 7
-; SI-NEXT: v_readlane_b32 s48, v22, 6
-; SI-NEXT: v_readlane_b32 s39, v22, 5
-; SI-NEXT: v_readlane_b32 s38, v22, 4
+; SI-NEXT: s_or_b32 s15, s15, s26
+; SI-NEXT: v_readlane_b32 s30, v22, 4
+; SI-NEXT: v_mov_b32_e32 v0, s12
+; SI-NEXT: v_mov_b32_e32 v1, s13
+; SI-NEXT: v_mov_b32_e32 v2, s10
+; SI-NEXT: v_mov_b32_e32 v3, s11
+; SI-NEXT: v_mov_b32_e32 v4, s8
+; SI-NEXT: v_mov_b32_e32 v5, s9
+; SI-NEXT: v_mov_b32_e32 v6, s6
+; SI-NEXT: v_mov_b32_e32 v7, s7
+; SI-NEXT: v_mov_b32_e32 v8, s4
+; SI-NEXT: v_mov_b32_e32 v9, s5
+; SI-NEXT: v_mov_b32_e32 v10, s16
+; SI-NEXT: v_mov_b32_e32 v11, s17
+; SI-NEXT: v_mov_b32_e32 v12, s18
+; SI-NEXT: v_mov_b32_e32 v13, s19
+; SI-NEXT: v_mov_b32_e32 v14, s20
+; SI-NEXT: v_mov_b32_e32 v15, s21
+; SI-NEXT: v_mov_b32_e32 v16, s22
+; SI-NEXT: v_mov_b32_e32 v17, s23
+; SI-NEXT: v_mov_b32_e32 v18, s24
+; SI-NEXT: v_mov_b32_e32 v19, s25
+; SI-NEXT: v_mov_b32_e32 v20, s14
+; SI-NEXT: v_mov_b32_e32 v21, s15
+; SI-NEXT: v_readlane_b32 s31, v22, 5
; SI-NEXT: v_readlane_b32 s37, v22, 3
; SI-NEXT: v_readlane_b32 s36, v22, 2
; SI-NEXT: v_readlane_b32 s35, v22, 1
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
index ddee91e20f7dd..a40b442cacbb1 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
@@ -13779,10 +13779,10 @@ define inreg <48 x i16> @bitcast_v24f32_to_v48i16_scalar(<24 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
@@ -17633,10 +17633,10 @@ define inreg <48 x half> @bitcast_v24f32_to_v48f16_scalar(<24 x float> inreg %a,
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
+; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v28
; SI-NEXT: v_and_b32_e32 v16, 0xffff, v16
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
; SI-NEXT: v_and_b32_e32 v18, 0xffff, v18
@@ -29097,30 +29097,30 @@ define <48 x i16> @bitcast_v12f64_to_v48i16(<12 x double> %a, i32 %b) #0 {
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v24
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr55
-; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: ; implicit-def: $vgpr54
-; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: ; implicit-def: $vgpr53
-; SI-NEXT: ; implicit-def: $vgpr33
+; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr52
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr39
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr26
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr36
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr24
-; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB48_2
@@ -29133,17 +29133,17 @@ define <48 x i16> @bitcast_v12f64_to_v48i16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v29, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v30, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v31, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v33, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v36, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v39, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v50, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v23
+; SI-NEXT: v_alignbit_b32 v32, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v35, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v38, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v49, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v23
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v19
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v5
@@ -29173,17 +29173,17 @@ define <48 x i16> @bitcast_v12f64_to_v48i16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v29, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v30, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v31, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v33, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v36, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v39, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v50, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v23
+; SI-NEXT: v_alignbit_b32 v32, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v35, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v38, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v49, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v23
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v19
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v5
@@ -29191,18 +29191,18 @@ define <48 x i16> @bitcast_v12f64_to_v48i16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v1
; SI-NEXT: .LBB48_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v50
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v49
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v38
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
@@ -29215,46 +29215,46 @@ define <48 x i16> @bitcast_v12f64_to_v48i16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
-; SI-NEXT: v_or_b32_e32 v0, v0, v50
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v55
-; SI-NEXT: v_or_b32_e32 v2, v2, v39
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v55
+; SI-NEXT: v_or_b32_e32 v2, v2, v38
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v54
-; SI-NEXT: v_or_b32_e32 v4, v4, v36
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v54
+; SI-NEXT: v_or_b32_e32 v4, v4, v35
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v53
-; SI-NEXT: v_or_b32_e32 v6, v6, v33
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v53
+; SI-NEXT: v_or_b32_e32 v6, v6, v32
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v52
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v52
; SI-NEXT: v_or_b32_e32 v8, v8, v31
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v51
; SI-NEXT: v_or_b32_e32 v10, v10, v30
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v50
; SI-NEXT: v_or_b32_e32 v12, v12, v29
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v48
; SI-NEXT: v_or_b32_e32 v14, v14, v28
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v39
; SI-NEXT: v_or_b32_e32 v16, v16, v27
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v37
; SI-NEXT: v_or_b32_e32 v18, v18, v26
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
; SI-NEXT: v_or_b32_e32 v20, v20, v25
; SI-NEXT: v_and_b32_e32 v21, 0xffff, v21
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v34
; SI-NEXT: v_or_b32_e32 v22, v22, v24
; SI-NEXT: v_and_b32_e32 v23, 0xffff, v23
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v32
-; SI-NEXT: v_or_b32_e32 v1, v1, v50
-; SI-NEXT: v_or_b32_e32 v3, v3, v39
-; SI-NEXT: v_or_b32_e32 v5, v5, v36
-; SI-NEXT: v_or_b32_e32 v7, v7, v33
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v33
+; SI-NEXT: v_or_b32_e32 v1, v1, v49
+; SI-NEXT: v_or_b32_e32 v3, v3, v38
+; SI-NEXT: v_or_b32_e32 v5, v5, v35
+; SI-NEXT: v_or_b32_e32 v7, v7, v32
; SI-NEXT: v_or_b32_e32 v9, v9, v31
; SI-NEXT: v_or_b32_e32 v11, v11, v30
; SI-NEXT: v_or_b32_e32 v13, v13, v29
@@ -30055,12 +30055,12 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v14
; VI-NEXT: v_lshrrev_b32_e32 v36, 16, v13
; VI-NEXT: v_lshrrev_b32_e32 v33, 16, v12
-; VI-NEXT: v_lshrrev_b32_e32 v37, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v38, 16, v11
; VI-NEXT: v_lshrrev_b32_e32 v35, 16, v10
-; VI-NEXT: v_lshrrev_b32_e32 v39, 16, v9
-; VI-NEXT: v_lshrrev_b32_e32 v38, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v48, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v37, 16, v8
; VI-NEXT: v_lshrrev_b32_e32 v50, 16, v7
-; VI-NEXT: v_lshrrev_b32_e32 v48, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v39, 16, v6
; VI-NEXT: v_lshrrev_b32_e32 v52, 16, v5
; VI-NEXT: v_lshrrev_b32_e32 v49, 16, v4
; VI-NEXT: v_lshrrev_b32_e32 v54, 16, v3
@@ -30096,8 +30096,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; VI-NEXT: v_mov_b32_e32 v53, s79
; VI-NEXT: v_mov_b32_e32 v51, s78
; VI-NEXT: v_mov_b32_e32 v49, s77
-; VI-NEXT: v_mov_b32_e32 v48, s76
-; VI-NEXT: v_mov_b32_e32 v38, s75
+; VI-NEXT: v_mov_b32_e32 v39, s76
+; VI-NEXT: v_mov_b32_e32 v37, s75
; VI-NEXT: v_mov_b32_e32 v35, s74
; VI-NEXT: v_mov_b32_e32 v33, s73
; VI-NEXT: v_mov_b32_e32 v31, s72
@@ -30109,8 +30109,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; VI-NEXT: v_mov_b32_e32 v54, s58
; VI-NEXT: v_mov_b32_e32 v52, s57
; VI-NEXT: v_mov_b32_e32 v50, s56
-; VI-NEXT: v_mov_b32_e32 v39, s47
-; VI-NEXT: v_mov_b32_e32 v37, s46
+; VI-NEXT: v_mov_b32_e32 v48, s47
+; VI-NEXT: v_mov_b32_e32 v38, s46
; VI-NEXT: v_mov_b32_e32 v36, s45
; VI-NEXT: v_mov_b32_e32 v34, s44
; VI-NEXT: v_mov_b32_e32 v32, s43
@@ -30121,8 +30121,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; VI-NEXT: v_lshlrev_b32_e32 v53, 16, v53
; VI-NEXT: v_lshlrev_b32_e32 v51, 16, v51
; VI-NEXT: v_lshlrev_b32_e32 v49, 16, v49
-; VI-NEXT: v_lshlrev_b32_e32 v48, 16, v48
-; VI-NEXT: v_lshlrev_b32_e32 v38, 16, v38
+; VI-NEXT: v_lshlrev_b32_e32 v39, 16, v39
+; VI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -30136,12 +30136,12 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; VI-NEXT: v_lshlrev_b32_e32 v51, 16, v54
; VI-NEXT: v_or_b32_sdwa v4, v4, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v49, 16, v52
-; VI-NEXT: v_or_b32_sdwa v6, v6, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v48, 16, v50
-; VI-NEXT: v_or_b32_sdwa v8, v8, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v38, 16, v39
+; VI-NEXT: v_or_b32_sdwa v6, v6, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v39, 16, v50
+; VI-NEXT: v_or_b32_sdwa v8, v8, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v37, 16, v48
; VI-NEXT: v_or_b32_sdwa v10, v10, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v37
+; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; VI-NEXT: v_or_b32_sdwa v12, v12, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v36
; VI-NEXT: v_or_b32_sdwa v14, v14, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -30157,8 +30157,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; VI-NEXT: v_or_b32_sdwa v1, v1, v53 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v3, v3, v51 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v5, v5, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v7, v7, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v9, v9, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v7, v7, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v9, v9, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v11, v11, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v13, v13, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v15, v15, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -30267,12 +30267,12 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; GFX9-NEXT: v_lshrrev_b32_e32 v31, 16, v14
; GFX9-NEXT: v_lshrrev_b32_e32 v36, 16, v13
; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v37, 16, v11
+; GFX9-NEXT: v_lshrrev_b32_e32 v38, 16, v11
; GFX9-NEXT: v_lshrrev_b32_e32 v35, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v39, 16, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v38, 16, v8
+; GFX9-NEXT: v_lshrrev_b32_e32 v48, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v37, 16, v8
; GFX9-NEXT: v_lshrrev_b32_e32 v50, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v48, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v39, 16, v6
; GFX9-NEXT: v_lshrrev_b32_e32 v52, 16, v5
; GFX9-NEXT: v_lshrrev_b32_e32 v49, 16, v4
; GFX9-NEXT: v_lshrrev_b32_e32 v54, 16, v3
@@ -30308,8 +30308,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v53, s79
; GFX9-NEXT: v_mov_b32_e32 v51, s78
; GFX9-NEXT: v_mov_b32_e32 v49, s77
-; GFX9-NEXT: v_mov_b32_e32 v48, s76
-; GFX9-NEXT: v_mov_b32_e32 v38, s75
+; GFX9-NEXT: v_mov_b32_e32 v39, s76
+; GFX9-NEXT: v_mov_b32_e32 v37, s75
; GFX9-NEXT: v_mov_b32_e32 v35, s74
; GFX9-NEXT: v_mov_b32_e32 v33, s73
; GFX9-NEXT: v_mov_b32_e32 v31, s72
@@ -30321,8 +30321,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; GFX9-NEXT: v_mov_b32_e32 v54, s58
; GFX9-NEXT: v_mov_b32_e32 v52, s57
; GFX9-NEXT: v_mov_b32_e32 v50, s56
-; GFX9-NEXT: v_mov_b32_e32 v39, s47
-; GFX9-NEXT: v_mov_b32_e32 v37, s46
+; GFX9-NEXT: v_mov_b32_e32 v48, s47
+; GFX9-NEXT: v_mov_b32_e32 v38, s46
; GFX9-NEXT: v_mov_b32_e32 v36, s45
; GFX9-NEXT: v_mov_b32_e32 v34, s44
; GFX9-NEXT: v_mov_b32_e32 v32, s43
@@ -30333,8 +30333,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v53, 16, v53
; GFX9-NEXT: v_lshlrev_b32_e32 v51, 16, v51
; GFX9-NEXT: v_lshlrev_b32_e32 v49, 16, v49
-; GFX9-NEXT: v_lshlrev_b32_e32 v48, 16, v48
-; GFX9-NEXT: v_lshlrev_b32_e32 v38, 16, v38
+; GFX9-NEXT: v_lshlrev_b32_e32 v39, 16, v39
+; GFX9-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -30348,12 +30348,12 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; GFX9-NEXT: v_lshlrev_b32_e32 v51, 16, v54
; GFX9-NEXT: v_or_b32_sdwa v4, v49, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v49, 16, v52
-; GFX9-NEXT: v_or_b32_sdwa v6, v48, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v48, 16, v50
-; GFX9-NEXT: v_or_b32_sdwa v8, v38, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v38, 16, v39
+; GFX9-NEXT: v_or_b32_sdwa v6, v39, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v39, 16, v50
+; GFX9-NEXT: v_or_b32_sdwa v8, v37, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v37, 16, v48
; GFX9-NEXT: v_or_b32_sdwa v10, v35, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v37
+; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; GFX9-NEXT: v_or_b32_sdwa v12, v33, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v36
; GFX9-NEXT: v_or_b32_sdwa v14, v31, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -30369,8 +30369,8 @@ define inreg <48 x i16> @bitcast_v12f64_to_v48i16_scalar(<12 x double> inreg %a,
; GFX9-NEXT: v_or_b32_sdwa v1, v53, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v3, v51, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v5, v49, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v7, v48, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v9, v38, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v7, v39, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v9, v37, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v11, v35, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v13, v33, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v15, v31, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -32855,30 +32855,30 @@ define <48 x half> @bitcast_v12f64_to_v48f16(<12 x double> %a, i32 %b) #0 {
; SI: ; %bb.0:
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v24
-; SI-NEXT: ; implicit-def: $vgpr50
+; SI-NEXT: ; implicit-def: $vgpr49
; SI-NEXT: ; implicit-def: $vgpr55
-; SI-NEXT: ; implicit-def: $vgpr39
+; SI-NEXT: ; implicit-def: $vgpr38
; SI-NEXT: ; implicit-def: $vgpr54
-; SI-NEXT: ; implicit-def: $vgpr36
+; SI-NEXT: ; implicit-def: $vgpr35
; SI-NEXT: ; implicit-def: $vgpr53
-; SI-NEXT: ; implicit-def: $vgpr33
+; SI-NEXT: ; implicit-def: $vgpr32
; SI-NEXT: ; implicit-def: $vgpr52
; SI-NEXT: ; implicit-def: $vgpr31
; SI-NEXT: ; implicit-def: $vgpr51
; SI-NEXT: ; implicit-def: $vgpr30
-; SI-NEXT: ; implicit-def: $vgpr49
+; SI-NEXT: ; implicit-def: $vgpr50
; SI-NEXT: ; implicit-def: $vgpr29
; SI-NEXT: ; implicit-def: $vgpr48
; SI-NEXT: ; implicit-def: $vgpr28
-; SI-NEXT: ; implicit-def: $vgpr38
+; SI-NEXT: ; implicit-def: $vgpr39
; SI-NEXT: ; implicit-def: $vgpr27
; SI-NEXT: ; implicit-def: $vgpr37
; SI-NEXT: ; implicit-def: $vgpr26
-; SI-NEXT: ; implicit-def: $vgpr35
+; SI-NEXT: ; implicit-def: $vgpr36
; SI-NEXT: ; implicit-def: $vgpr25
; SI-NEXT: ; implicit-def: $vgpr34
; SI-NEXT: ; implicit-def: $vgpr24
-; SI-NEXT: ; implicit-def: $vgpr32
+; SI-NEXT: ; implicit-def: $vgpr33
; SI-NEXT: s_and_saveexec_b64 s[4:5], vcc
; SI-NEXT: s_xor_b64 s[4:5], exec, s[4:5]
; SI-NEXT: s_cbranch_execz .LBB52_2
@@ -32891,17 +32891,17 @@ define <48 x half> @bitcast_v12f64_to_v48f16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v29, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v30, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v31, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v33, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v36, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v39, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v50, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v23
+; SI-NEXT: v_alignbit_b32 v32, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v35, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v38, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v49, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v23
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v19
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v5
@@ -32931,17 +32931,17 @@ define <48 x half> @bitcast_v12f64_to_v48f16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_alignbit_b32 v29, v13, v12, 16
; SI-NEXT: v_alignbit_b32 v30, v11, v10, 16
; SI-NEXT: v_alignbit_b32 v31, v9, v8, 16
-; SI-NEXT: v_alignbit_b32 v33, v7, v6, 16
-; SI-NEXT: v_alignbit_b32 v36, v5, v4, 16
-; SI-NEXT: v_alignbit_b32 v39, v3, v2, 16
-; SI-NEXT: v_alignbit_b32 v50, v1, v0, 16
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v23
+; SI-NEXT: v_alignbit_b32 v32, v7, v6, 16
+; SI-NEXT: v_alignbit_b32 v35, v5, v4, 16
+; SI-NEXT: v_alignbit_b32 v38, v3, v2, 16
+; SI-NEXT: v_alignbit_b32 v49, v1, v0, 16
+; SI-NEXT: v_lshrrev_b32_e32 v33, 16, v23
; SI-NEXT: v_lshrrev_b32_e32 v34, 16, v21
-; SI-NEXT: v_lshrrev_b32_e32 v35, 16, v19
+; SI-NEXT: v_lshrrev_b32_e32 v36, 16, v19
; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v17
-; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v39, 16, v15
; SI-NEXT: v_lshrrev_b32_e32 v48, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v11
+; SI-NEXT: v_lshrrev_b32_e32 v50, 16, v11
; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v9
; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v7
; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v5
@@ -32949,18 +32949,18 @@ define <48 x half> @bitcast_v12f64_to_v48f16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v1
; SI-NEXT: .LBB52_4: ; %end
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v50
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v49
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v39
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v38
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v36
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v32
; SI-NEXT: v_and_b32_e32 v8, 0xffff, v8
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
+; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v30
; SI-NEXT: v_and_b32_e32 v12, 0xffff, v12
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v29
; SI-NEXT: v_and_b32_e32 v14, 0xffff, v14
@@ -32973,46 +32973,46 @@ define <48 x half> @bitcast_v12f64_to_v48f16(<12 x double> %a, i32 %b) #0 {
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
; SI-NEXT: v_and_b32_e32 v22, 0xffff, v22
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
-; SI-NEXT: v_or_b32_e32 v0, v0, v50
+; SI-NEXT: v_or_b32_e32 v0, v0, v49
; SI-NEXT: v_and_b32_e32 v1, 0xffff, v1
-; SI-NEXT: v_lshlrev_b32_e32 v50, 16, v55
-; SI-NEXT: v_or_b32_e32 v2, v2, v39
+; SI-NEXT: v_lshlrev_b32_e32 v49, 16, v55
+; SI-NEXT: v_or_b32_e32 v2, v2, v38
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v39, 16, v54
-; SI-NEXT: v_or_b32_e32 v4, v4, v36
+; SI-NEXT: v_lshlrev_b32_e32 v38, 16, v54
+; SI-NEXT: v_or_b32_e32 v4, v4, v35
; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
-; SI-NEXT: v_lshlrev_b32_e32 v36, 16, v53
-; SI-NEXT: v_or_b32_e32 v6, v6, v33
+; SI-NEXT: v_lshlrev_b32_e32 v35, 16, v53
+; SI-NEXT: v_or_b32_e32 v6, v6, v32
; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
-; SI-NEXT: v_lshlrev_b32_e32 v33, 16, v52
+; SI-NEXT: v_lshlrev_b32_e32 v32, 16, v52
; SI-NEXT: v_or_b32_e32 v8, v8, v31
; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: v_lshlrev_b32_e32 v31, 16, v51
; SI-NEXT: v_or_b32_e32 v10, v10, v30
; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v49
+; SI-NEXT: v_lshlrev_b32_e32 v30, 16, v50
; SI-NEXT: v_or_b32_e32 v12, v12, v29
; SI-NEXT: v_and_b32_e32 v13, 0xffff, v13
; SI-NEXT: v_lshlrev_b32_e32 v29, 16, v48
; SI-NEXT: v_or_b32_e32 v14, v14, v28
; SI-NEXT: v_and_b32_e32 v15, 0xffff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v38
+; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v39
; SI-NEXT: v_or_b32_e32 v16, v16, v27
; SI-NEXT: v_and_b32_e32 v17, 0xffff, v17
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v37
; SI-NEXT: v_or_b32_e32 v18, v18, v26
; SI-NEXT: v_and_b32_e32 v19, 0xffff, v19
-; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v35
+; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v36
; SI-NEXT: v_or_b32_e32 v20, v20, v25
; SI-NEXT: v_and_b32_e32 v21, 0xffff, v21
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v34
; SI-NEXT: v_or_b32_e32 v22, v22, v24
; SI-NEXT: v_and_b32_e32 v23, 0xffff, v23
-; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v32
-; SI-NEXT: v_or_b32_e32 v1, v1, v50
-; SI-NEXT: v_or_b32_e32 v3, v3, v39
-; SI-NEXT: v_or_b32_e32 v5, v5, v36
-; SI-NEXT: v_or_b32_e32 v7, v7, v33
+; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v33
+; SI-NEXT: v_or_b32_e32 v1, v1, v49
+; SI-NEXT: v_or_b32_e32 v3, v3, v38
+; SI-NEXT: v_or_b32_e32 v5, v5, v35
+; SI-NEXT: v_or_b32_e32 v7, v7, v32
; SI-NEXT: v_or_b32_e32 v9, v9, v31
; SI-NEXT: v_or_b32_e32 v11, v11, v30
; SI-NEXT: v_or_b32_e32 v13, v13, v29
@@ -33813,12 +33813,12 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; VI-NEXT: v_lshrrev_b32_e32 v31, 16, v14
; VI-NEXT: v_lshrrev_b32_e32 v36, 16, v13
; VI-NEXT: v_lshrrev_b32_e32 v33, 16, v12
-; VI-NEXT: v_lshrrev_b32_e32 v37, 16, v11
+; VI-NEXT: v_lshrrev_b32_e32 v38, 16, v11
; VI-NEXT: v_lshrrev_b32_e32 v35, 16, v10
-; VI-NEXT: v_lshrrev_b32_e32 v39, 16, v9
-; VI-NEXT: v_lshrrev_b32_e32 v38, 16, v8
+; VI-NEXT: v_lshrrev_b32_e32 v48, 16, v9
+; VI-NEXT: v_lshrrev_b32_e32 v37, 16, v8
; VI-NEXT: v_lshrrev_b32_e32 v50, 16, v7
-; VI-NEXT: v_lshrrev_b32_e32 v48, 16, v6
+; VI-NEXT: v_lshrrev_b32_e32 v39, 16, v6
; VI-NEXT: v_lshrrev_b32_e32 v52, 16, v5
; VI-NEXT: v_lshrrev_b32_e32 v49, 16, v4
; VI-NEXT: v_lshrrev_b32_e32 v54, 16, v3
@@ -33854,8 +33854,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; VI-NEXT: v_mov_b32_e32 v53, s79
; VI-NEXT: v_mov_b32_e32 v51, s78
; VI-NEXT: v_mov_b32_e32 v49, s77
-; VI-NEXT: v_mov_b32_e32 v48, s76
-; VI-NEXT: v_mov_b32_e32 v38, s75
+; VI-NEXT: v_mov_b32_e32 v39, s76
+; VI-NEXT: v_mov_b32_e32 v37, s75
; VI-NEXT: v_mov_b32_e32 v35, s74
; VI-NEXT: v_mov_b32_e32 v33, s73
; VI-NEXT: v_mov_b32_e32 v31, s72
@@ -33867,8 +33867,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; VI-NEXT: v_mov_b32_e32 v54, s58
; VI-NEXT: v_mov_b32_e32 v52, s57
; VI-NEXT: v_mov_b32_e32 v50, s56
-; VI-NEXT: v_mov_b32_e32 v39, s47
-; VI-NEXT: v_mov_b32_e32 v37, s46
+; VI-NEXT: v_mov_b32_e32 v48, s47
+; VI-NEXT: v_mov_b32_e32 v38, s46
; VI-NEXT: v_mov_b32_e32 v36, s45
; VI-NEXT: v_mov_b32_e32 v34, s44
; VI-NEXT: v_mov_b32_e32 v32, s43
@@ -33879,8 +33879,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; VI-NEXT: v_lshlrev_b32_e32 v53, 16, v53
; VI-NEXT: v_lshlrev_b32_e32 v51, 16, v51
; VI-NEXT: v_lshlrev_b32_e32 v49, 16, v49
-; VI-NEXT: v_lshlrev_b32_e32 v48, 16, v48
-; VI-NEXT: v_lshlrev_b32_e32 v38, 16, v38
+; VI-NEXT: v_lshlrev_b32_e32 v39, 16, v39
+; VI-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; VI-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -33894,12 +33894,12 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; VI-NEXT: v_lshlrev_b32_e32 v51, 16, v54
; VI-NEXT: v_or_b32_sdwa v4, v4, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v49, 16, v52
-; VI-NEXT: v_or_b32_sdwa v6, v6, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v48, 16, v50
-; VI-NEXT: v_or_b32_sdwa v8, v8, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v38, 16, v39
+; VI-NEXT: v_or_b32_sdwa v6, v6, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v39, 16, v50
+; VI-NEXT: v_or_b32_sdwa v8, v8, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_lshlrev_b32_e32 v37, 16, v48
; VI-NEXT: v_or_b32_sdwa v10, v10, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v37
+; VI-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; VI-NEXT: v_or_b32_sdwa v12, v12, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_lshlrev_b32_e32 v33, 16, v36
; VI-NEXT: v_or_b32_sdwa v14, v14, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -33915,8 +33915,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; VI-NEXT: v_or_b32_sdwa v1, v1, v53 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v3, v3, v51 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v5, v5, v49 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v7, v7, v48 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; VI-NEXT: v_or_b32_sdwa v9, v9, v38 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v7, v7, v39 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; VI-NEXT: v_or_b32_sdwa v9, v9, v37 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v11, v11, v35 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v13, v13, v33 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v15, v15, v31 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
@@ -34025,12 +34025,12 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; GFX9-NEXT: v_lshrrev_b32_e32 v31, 16, v14
; GFX9-NEXT: v_lshrrev_b32_e32 v36, 16, v13
; GFX9-NEXT: v_lshrrev_b32_e32 v33, 16, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v37, 16, v11
+; GFX9-NEXT: v_lshrrev_b32_e32 v38, 16, v11
; GFX9-NEXT: v_lshrrev_b32_e32 v35, 16, v10
-; GFX9-NEXT: v_lshrrev_b32_e32 v39, 16, v9
-; GFX9-NEXT: v_lshrrev_b32_e32 v38, 16, v8
+; GFX9-NEXT: v_lshrrev_b32_e32 v48, 16, v9
+; GFX9-NEXT: v_lshrrev_b32_e32 v37, 16, v8
; GFX9-NEXT: v_lshrrev_b32_e32 v50, 16, v7
-; GFX9-NEXT: v_lshrrev_b32_e32 v48, 16, v6
+; GFX9-NEXT: v_lshrrev_b32_e32 v39, 16, v6
; GFX9-NEXT: v_lshrrev_b32_e32 v52, 16, v5
; GFX9-NEXT: v_lshrrev_b32_e32 v49, 16, v4
; GFX9-NEXT: v_lshrrev_b32_e32 v54, 16, v3
@@ -34066,8 +34066,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; GFX9-NEXT: v_mov_b32_e32 v53, s79
; GFX9-NEXT: v_mov_b32_e32 v51, s78
; GFX9-NEXT: v_mov_b32_e32 v49, s77
-; GFX9-NEXT: v_mov_b32_e32 v48, s76
-; GFX9-NEXT: v_mov_b32_e32 v38, s75
+; GFX9-NEXT: v_mov_b32_e32 v39, s76
+; GFX9-NEXT: v_mov_b32_e32 v37, s75
; GFX9-NEXT: v_mov_b32_e32 v35, s74
; GFX9-NEXT: v_mov_b32_e32 v33, s73
; GFX9-NEXT: v_mov_b32_e32 v31, s72
@@ -34079,8 +34079,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; GFX9-NEXT: v_mov_b32_e32 v54, s58
; GFX9-NEXT: v_mov_b32_e32 v52, s57
; GFX9-NEXT: v_mov_b32_e32 v50, s56
-; GFX9-NEXT: v_mov_b32_e32 v39, s47
-; GFX9-NEXT: v_mov_b32_e32 v37, s46
+; GFX9-NEXT: v_mov_b32_e32 v48, s47
+; GFX9-NEXT: v_mov_b32_e32 v38, s46
; GFX9-NEXT: v_mov_b32_e32 v36, s45
; GFX9-NEXT: v_mov_b32_e32 v34, s44
; GFX9-NEXT: v_mov_b32_e32 v32, s43
@@ -34091,8 +34091,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; GFX9-NEXT: v_lshlrev_b32_e32 v53, 16, v53
; GFX9-NEXT: v_lshlrev_b32_e32 v51, 16, v51
; GFX9-NEXT: v_lshlrev_b32_e32 v49, 16, v49
-; GFX9-NEXT: v_lshlrev_b32_e32 v48, 16, v48
-; GFX9-NEXT: v_lshlrev_b32_e32 v38, 16, v38
+; GFX9-NEXT: v_lshlrev_b32_e32 v39, 16, v39
+; GFX9-NEXT: v_lshlrev_b32_e32 v37, 16, v37
; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v35
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v33
; GFX9-NEXT: v_lshlrev_b32_e32 v31, 16, v31
@@ -34106,12 +34106,12 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; GFX9-NEXT: v_lshlrev_b32_e32 v51, 16, v54
; GFX9-NEXT: v_or_b32_sdwa v4, v49, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v49, 16, v52
-; GFX9-NEXT: v_or_b32_sdwa v6, v48, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v48, 16, v50
-; GFX9-NEXT: v_or_b32_sdwa v8, v38, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v38, 16, v39
+; GFX9-NEXT: v_or_b32_sdwa v6, v39, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v39, 16, v50
+; GFX9-NEXT: v_or_b32_sdwa v8, v37, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_lshlrev_b32_e32 v37, 16, v48
; GFX9-NEXT: v_or_b32_sdwa v10, v35, v10 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v37
+; GFX9-NEXT: v_lshlrev_b32_e32 v35, 16, v38
; GFX9-NEXT: v_or_b32_sdwa v12, v33, v12 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_lshlrev_b32_e32 v33, 16, v36
; GFX9-NEXT: v_or_b32_sdwa v14, v31, v14 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -34127,8 +34127,8 @@ define inreg <48 x half> @bitcast_v12f64_to_v48f16_scalar(<12 x double> inreg %a
; GFX9-NEXT: v_or_b32_sdwa v1, v53, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v3, v51, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v5, v49, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v7, v48, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
-; GFX9-NEXT: v_or_b32_sdwa v9, v38, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v7, v39, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
+; GFX9-NEXT: v_or_b32_sdwa v9, v37, v9 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v11, v35, v11 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v13, v33, v13 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
; GFX9-NEXT: v_or_b32_sdwa v15, v31, v15 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
@@ -39583,6 +39583,7 @@ define <48 x i16> @bitcast_v48f16_to_v48i16(<48 x half> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v22, v54, v22, s6
; GFX9-NEXT: v_perm_b32 v21, v53, v21, s6
; GFX9-NEXT: v_perm_b32 v20, v52, v20, s6
+; GFX9-NEXT: s_movk_i32 s7, 0x200
; GFX9-NEXT: v_perm_b32 v19, v51, v19, s6
; GFX9-NEXT: v_perm_b32 v18, v50, v18, s6
; GFX9-NEXT: v_perm_b32 v17, v49, v17, s6
@@ -39590,7 +39591,6 @@ define <48 x i16> @bitcast_v48f16_to_v48i16(<48 x half> %a, i32 %b) #0 {
; GFX9-NEXT: v_perm_b32 v15, v39, v15, s6
; GFX9-NEXT: v_perm_b32 v14, v38, v14, s6
; GFX9-NEXT: v_perm_b32 v13, v37, v13, s6
-; GFX9-NEXT: s_movk_i32 s7, 0x200
; GFX9-NEXT: v_perm_b32 v12, v36, v12, s6
; GFX9-NEXT: v_perm_b32 v11, v35, v11, s6
; GFX9-NEXT: v_perm_b32 v10, v34, v10, s6
diff --git a/llvm/test/CodeGen/AMDGPU/bf16.ll b/llvm/test/CodeGen/AMDGPU/bf16.ll
index 14dfc9d8314e9..92ef04e0b7d2b 100644
--- a/llvm/test/CodeGen/AMDGPU/bf16.ll
+++ b/llvm/test/CodeGen/AMDGPU/bf16.ll
@@ -666,33 +666,33 @@ define <64 x bfloat> @v_load_global_v64bf16(ptr addrspace(1) %ptr) #0 {
; GFX8-LABEL: v_load_global_v64bf16:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-NEXT: v_mov_b32_e32 v28, v0
-; GFX8-NEXT: v_mov_b32_e32 v29, v1
-; GFX8-NEXT: v_add_u32_e32 v4, vcc, 16, v28
-; GFX8-NEXT: v_addc_u32_e32 v5, vcc, 0, v29, vcc
-; GFX8-NEXT: v_add_u32_e32 v8, vcc, 32, v28
-; GFX8-NEXT: v_addc_u32_e32 v9, vcc, 0, v29, vcc
-; GFX8-NEXT: v_add_u32_e32 v12, vcc, 48, v28
-; GFX8-NEXT: v_addc_u32_e32 v13, vcc, 0, v29, vcc
-; GFX8-NEXT: v_add_u32_e32 v16, vcc, 64, v28
-; GFX8-NEXT: v_addc_u32_e32 v17, vcc, 0, v29, vcc
+; GFX8-NEXT: v_mov_b32_e32 v20, v0
+; GFX8-NEXT: v_mov_b32_e32 v21, v1
+; GFX8-NEXT: v_add_u32_e32 v4, vcc, 16, v20
+; GFX8-NEXT: v_addc_u32_e32 v5, vcc, 0, v21, vcc
+; GFX8-NEXT: v_add_u32_e32 v8, vcc, 32, v20
+; GFX8-NEXT: v_addc_u32_e32 v9, vcc, 0, v21, vcc
+; GFX8-NEXT: v_add_u32_e32 v12, vcc, 48, v20
+; GFX8-NEXT: v_addc_u32_e32 v13, vcc, 0, v21, vcc
+; GFX8-NEXT: v_add_u32_e32 v16, vcc, 64, v20
+; GFX8-NEXT: v_addc_u32_e32 v17, vcc, 0, v21, vcc
; GFX8-NEXT: s_movk_i32 s4, 0x50
-; GFX8-NEXT: v_add_u32_e32 v20, vcc, s4, v28
-; GFX8-NEXT: v_addc_u32_e32 v21, vcc, 0, v29, vcc
+; GFX8-NEXT: v_add_u32_e32 v22, vcc, s4, v20
+; GFX8-NEXT: v_addc_u32_e32 v23, vcc, 0, v21, vcc
; GFX8-NEXT: s_movk_i32 s4, 0x60
-; GFX8-NEXT: v_add_u32_e32 v24, vcc, s4, v28
-; GFX8-NEXT: v_addc_u32_e32 v25, vcc, 0, v29, vcc
+; GFX8-NEXT: v_add_u32_e32 v24, vcc, s4, v20
+; GFX8-NEXT: v_addc_u32_e32 v25, vcc, 0, v21, vcc
; GFX8-NEXT: s_movk_i32 s4, 0x70
-; GFX8-NEXT: flat_load_dwordx4 v[0:3], v[28:29]
-; GFX8-NEXT: v_add_u32_e32 v28, vcc, s4, v28
-; GFX8-NEXT: v_addc_u32_e32 v29, vcc, 0, v29, vcc
+; GFX8-NEXT: v_add_u32_e32 v28, vcc, s4, v20
+; GFX8-NEXT: v_addc_u32_e32 v29, vcc, 0, v21, vcc
+; GFX8-NEXT: flat_load_dwordx4 v[0:3], v[20:21]
+; GFX8-NEXT: flat_load_dwordx4 v[20:23], v[22:23]
+; GFX8-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
; GFX8-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
; GFX8-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
-; GFX8-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
-; GFX8-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
-; GFX8-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
-; GFX8-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
@@ -49096,29 +49096,29 @@ define <32 x bfloat> @v_fma_v32bf16(<32 x bfloat> %a, <32 x bfloat> %b, <32 x bf
; GFX8-NEXT: v_lshlrev_b32_e32 v20, 16, v33
; GFX8-NEXT: v_and_b32_e32 v33, 0xffff0000, v33
; GFX8-NEXT: v_fma_f32 v3, v3, v19, v33
-; GFX8-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:12
+; GFX8-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:12
; GFX8-NEXT: v_fma_f32 v20, v35, v34, v20
; GFX8-NEXT: v_lshlrev_b32_e32 v34, 16, v18
; GFX8-NEXT: v_lshlrev_b32_e32 v35, 16, v2
; GFX8-NEXT: v_and_b32_e32 v18, 0xffff0000, v18
; GFX8-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_lshlrev_b32_e32 v19, 16, v33
-; GFX8-NEXT: v_and_b32_e32 v33, 0xffff0000, v33
-; GFX8-NEXT: v_fma_f32 v2, v2, v18, v33
+; GFX8-NEXT: v_lshlrev_b32_e32 v33, 16, v19
+; GFX8-NEXT: v_and_b32_e32 v19, 0xffff0000, v19
+; GFX8-NEXT: v_fma_f32 v2, v2, v18, v19
; GFX8-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:8
-; GFX8-NEXT: v_fma_f32 v19, v35, v34, v19
+; GFX8-NEXT: v_fma_f32 v33, v35, v34, v33
; GFX8-NEXT: v_lshlrev_b32_e32 v34, 16, v17
; GFX8-NEXT: v_lshlrev_b32_e32 v35, 16, v1
; GFX8-NEXT: v_and_b32_e32 v17, 0xffff0000, v17
; GFX8-NEXT: v_and_b32_e32 v1, 0xffff0000, v1
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_lshlrev_b32_e32 v33, 16, v18
+; GFX8-NEXT: v_lshlrev_b32_e32 v19, 16, v18
; GFX8-NEXT: v_and_b32_e32 v18, 0xffff0000, v18
; GFX8-NEXT: v_fma_f32 v1, v1, v17, v18
; GFX8-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:4
-; GFX8-NEXT: v_fma_f32 v33, v35, v34, v33
-; GFX8-NEXT: v_lshlrev_b32_e32 v34, 16, v16
+; GFX8-NEXT: v_fma_f32 v34, v35, v34, v19
+; GFX8-NEXT: v_lshlrev_b32_e32 v19, 16, v16
; GFX8-NEXT: v_lshlrev_b32_e32 v35, 16, v0
; GFX8-NEXT: v_and_b32_e32 v16, 0xffff0000, v16
; GFX8-NEXT: v_and_b32_e32 v0, 0xffff0000, v0
@@ -49141,7 +49141,7 @@ define <32 x bfloat> @v_fma_v32bf16(<32 x bfloat> %a, <32 x bfloat> %b, <32 x bf
; GFX8-NEXT: v_bfe_u32 v15, v32, 16, 1
; GFX8-NEXT: v_add_u32_e32 v15, vcc, v15, v32
; GFX8-NEXT: v_add_u32_e32 v15, vcc, s4, v15
-; GFX8-NEXT: v_fma_f32 v34, v35, v34, v18
+; GFX8-NEXT: v_fma_f32 v35, v35, v19, v18
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v32, v32
; GFX8-NEXT: v_or_b32_e32 v18, 0x400000, v32
; GFX8-NEXT: v_cndmask_b32_e32 v15, v15, v18, vcc
@@ -49155,14 +49155,14 @@ define <32 x bfloat> @v_fma_v32bf16(<32 x bfloat> %a, <32 x bfloat> %b, <32 x bf
; GFX8-NEXT: v_add_u32_e32 v14, vcc, v14, v30
; GFX8-NEXT: v_add_u32_e32 v14, vcc, s4, v14
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v30, v30
-; GFX8-NEXT: v_or_b32_e32 v30, 0x400000, v30
-; GFX8-NEXT: v_cndmask_b32_e32 v14, v14, v30, vcc
-; GFX8-NEXT: v_bfe_u32 v30, v13, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v30, vcc, v30, v13
-; GFX8-NEXT: v_add_u32_e32 v30, vcc, s4, v30
+; GFX8-NEXT: v_or_b32_e32 v19, 0x400000, v30
+; GFX8-NEXT: v_cndmask_b32_e32 v14, v14, v19, vcc
+; GFX8-NEXT: v_bfe_u32 v19, v13, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v19, vcc, v19, v13
+; GFX8-NEXT: v_add_u32_e32 v19, vcc, s4, v19
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v13, v13
; GFX8-NEXT: v_or_b32_e32 v13, 0x400000, v13
-; GFX8-NEXT: v_cndmask_b32_e32 v30, v30, v13, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v19, v19, v13, vcc
; GFX8-NEXT: v_bfe_u32 v13, v29, 16, 1
; GFX8-NEXT: v_add_u32_e32 v13, vcc, v13, v29
; GFX8-NEXT: v_add_u32_e32 v13, vcc, s4, v13
@@ -49174,144 +49174,144 @@ define <32 x bfloat> @v_fma_v32bf16(<32 x bfloat> %a, <32 x bfloat> %b, <32 x bf
; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v12, v12
; GFX8-NEXT: v_or_b32_e32 v12, 0x400000, v12
-; GFX8-NEXT: v_cndmask_b32_e32 v29, v29, v12, vcc
-; GFX8-NEXT: v_bfe_u32 v12, v28, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v12, vcc, v12, v28
-; GFX8-NEXT: v_add_u32_e32 v12, vcc, s4, v12
+; GFX8-NEXT: v_cndmask_b32_e32 v12, v29, v12, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v28, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v28
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v28, v28
; GFX8-NEXT: v_or_b32_e32 v28, 0x400000, v28
-; GFX8-NEXT: v_cndmask_b32_e32 v12, v12, v28, vcc
-; GFX8-NEXT: v_bfe_u32 v28, v11, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v28, vcc, v28, v11
-; GFX8-NEXT: v_add_u32_e32 v28, vcc, s4, v28
+; GFX8-NEXT: v_cndmask_b32_e32 v28, v29, v28, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v11, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v11
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v11, v11
; GFX8-NEXT: v_or_b32_e32 v11, 0x400000, v11
-; GFX8-NEXT: v_cndmask_b32_e32 v28, v28, v11, vcc
-; GFX8-NEXT: v_bfe_u32 v11, v27, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v11, vcc, v11, v27
-; GFX8-NEXT: v_add_u32_e32 v11, vcc, s4, v11
+; GFX8-NEXT: v_cndmask_b32_e32 v11, v29, v11, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v27, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v27
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v27, v27
; GFX8-NEXT: v_or_b32_e32 v27, 0x400000, v27
-; GFX8-NEXT: v_cndmask_b32_e32 v11, v11, v27, vcc
-; GFX8-NEXT: v_bfe_u32 v27, v10, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v27, vcc, v27, v10
-; GFX8-NEXT: v_add_u32_e32 v27, vcc, s4, v27
+; GFX8-NEXT: v_cndmask_b32_e32 v27, v29, v27, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v10, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v10
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v10, v10
; GFX8-NEXT: v_or_b32_e32 v10, 0x400000, v10
-; GFX8-NEXT: v_cndmask_b32_e32 v27, v27, v10, vcc
-; GFX8-NEXT: v_bfe_u32 v10, v26, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v10, vcc, v10, v26
-; GFX8-NEXT: v_add_u32_e32 v10, vcc, s4, v10
+; GFX8-NEXT: v_cndmask_b32_e32 v10, v29, v10, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v26, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v26
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v26, v26
; GFX8-NEXT: v_or_b32_e32 v26, 0x400000, v26
-; GFX8-NEXT: v_cndmask_b32_e32 v10, v10, v26, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v9, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v9
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v26, v29, v26, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v9, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v9
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v9, v9
; GFX8-NEXT: v_or_b32_e32 v9, 0x400000, v9
-; GFX8-NEXT: v_cndmask_b32_e32 v9, v26, v9, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v25, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v25
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v29, v9, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v25, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v25
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v25, v25
; GFX8-NEXT: v_or_b32_e32 v25, 0x400000, v25
-; GFX8-NEXT: v_cndmask_b32_e32 v25, v26, v25, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v8, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v8
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v25, v29, v25, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v8, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v8
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v8, v8
; GFX8-NEXT: v_or_b32_e32 v8, 0x400000, v8
-; GFX8-NEXT: v_cndmask_b32_e32 v8, v26, v8, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v24, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v24
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v8, v29, v8, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v24, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v24
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v24, v24
; GFX8-NEXT: v_or_b32_e32 v24, 0x400000, v24
-; GFX8-NEXT: v_cndmask_b32_e32 v24, v26, v24, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v7, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v7
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v24, v29, v24, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v7, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v7
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v7, v7
; GFX8-NEXT: v_or_b32_e32 v7, 0x400000, v7
-; GFX8-NEXT: v_cndmask_b32_e32 v7, v26, v7, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v23, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v23
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v7, v29, v7, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v23, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v23
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v23, v23
; GFX8-NEXT: v_or_b32_e32 v23, 0x400000, v23
-; GFX8-NEXT: v_cndmask_b32_e32 v23, v26, v23, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v6, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v6
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v23, v29, v23, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v6, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v6
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v6, v6
; GFX8-NEXT: v_or_b32_e32 v6, 0x400000, v6
-; GFX8-NEXT: v_cndmask_b32_e32 v6, v26, v6, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v22, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v22
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v6, v29, v6, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v22, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v22
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v22, v22
; GFX8-NEXT: v_or_b32_e32 v22, 0x400000, v22
-; GFX8-NEXT: v_cndmask_b32_e32 v22, v26, v22, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v5, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v5
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v22, v29, v22, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v5, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v5
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v5, v5
; GFX8-NEXT: v_or_b32_e32 v5, 0x400000, v5
-; GFX8-NEXT: v_cndmask_b32_e32 v5, v26, v5, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v21, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v21
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v5, v29, v5, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v21, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v21
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v21, v21
; GFX8-NEXT: v_or_b32_e32 v21, 0x400000, v21
-; GFX8-NEXT: v_cndmask_b32_e32 v21, v26, v21, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v4, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v4
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v21, v29, v21, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v4, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v4
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v4, v4
; GFX8-NEXT: v_or_b32_e32 v4, 0x400000, v4
-; GFX8-NEXT: v_cndmask_b32_e32 v4, v26, v4, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v20, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v20
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v4, v29, v4, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v20, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v20
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v20, v20
; GFX8-NEXT: v_or_b32_e32 v20, 0x400000, v20
-; GFX8-NEXT: v_cndmask_b32_e32 v20, v26, v20, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v3, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v3
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v20, v29, v20, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v3, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v3
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v3, v3
; GFX8-NEXT: v_or_b32_e32 v3, 0x400000, v3
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v26, v3, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v19, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v19
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
-; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v19, v19
-; GFX8-NEXT: v_or_b32_e32 v19, 0x400000, v19
-; GFX8-NEXT: v_cndmask_b32_e32 v19, v26, v19, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v2, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v2
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v3, v29, v3, vcc
+; GFX8-NEXT: v_bfe_u32 v29, v33, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, v29, v33
+; GFX8-NEXT: v_add_u32_e32 v29, vcc, s4, v29
+; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v33, v33
+; GFX8-NEXT: v_or_b32_e32 v30, 0x400000, v33
+; GFX8-NEXT: v_cndmask_b32_e32 v29, v29, v30, vcc
+; GFX8-NEXT: v_bfe_u32 v30, v2, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v30, vcc, v30, v2
+; GFX8-NEXT: v_add_u32_e32 v30, vcc, s4, v30
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v2, v2
; GFX8-NEXT: v_or_b32_e32 v2, 0x400000, v2
-; GFX8-NEXT: v_cndmask_b32_e32 v2, v26, v2, vcc
-; GFX8-NEXT: v_bfe_u32 v26, v33, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, v26, v33
-; GFX8-NEXT: v_add_u32_e32 v26, vcc, s4, v26
-; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v33, v33
-; GFX8-NEXT: v_or_b32_e32 v31, 0x400000, v33
-; GFX8-NEXT: v_cndmask_b32_e32 v26, v26, v31, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v2, v30, v2, vcc
+; GFX8-NEXT: v_bfe_u32 v30, v34, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v30, vcc, v30, v34
+; GFX8-NEXT: v_add_u32_e32 v30, vcc, s4, v30
+; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v34, v34
+; GFX8-NEXT: v_or_b32_e32 v31, 0x400000, v34
+; GFX8-NEXT: v_cndmask_b32_e32 v30, v30, v31, vcc
; GFX8-NEXT: v_bfe_u32 v31, v1, 16, 1
; GFX8-NEXT: v_add_u32_e32 v31, vcc, v31, v1
; GFX8-NEXT: v_add_u32_e32 v31, vcc, s4, v31
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v1, v1
; GFX8-NEXT: v_or_b32_e32 v1, 0x400000, v1
; GFX8-NEXT: v_cndmask_b32_e32 v1, v31, v1, vcc
-; GFX8-NEXT: v_bfe_u32 v31, v34, 16, 1
-; GFX8-NEXT: v_add_u32_e32 v31, vcc, v31, v34
+; GFX8-NEXT: v_bfe_u32 v31, v35, 16, 1
+; GFX8-NEXT: v_add_u32_e32 v31, vcc, v31, v35
; GFX8-NEXT: v_add_u32_e32 v31, vcc, s4, v31
-; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v34, v34
-; GFX8-NEXT: v_or_b32_e32 v32, 0x400000, v34
+; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v35, v35
+; GFX8-NEXT: v_or_b32_e32 v32, 0x400000, v35
; GFX8-NEXT: v_cndmask_b32_e32 v31, v31, v32, vcc
; GFX8-NEXT: v_bfe_u32 v32, v0, 16, 1
; GFX8-NEXT: v_add_u32_e32 v32, vcc, v32, v0
@@ -49319,35 +49319,35 @@ define <32 x bfloat> @v_fma_v32bf16(<32 x bfloat> %a, <32 x bfloat> %b, <32 x bf
; GFX8-NEXT: v_cmp_u_f32_e32 vcc, v0, v0
; GFX8-NEXT: v_or_b32_e32 v0, 0x400000, v0
; GFX8-NEXT: v_cndmask_b32_e32 v0, v32, v0, vcc
+; GFX8-NEXT: v_lshrrev_b32_e32 v0, 16, v0
+; GFX8-NEXT: v_lshrrev_b32_e32 v1, 16, v1
; GFX8-NEXT: v_lshrrev_b32_e32 v2, 16, v2
; GFX8-NEXT: v_lshrrev_b32_e32 v3, 16, v3
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 16, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v5, 16, v5
-; GFX8-NEXT: v_lshrrev_b32_e32 v0, 16, v0
-; GFX8-NEXT: v_lshrrev_b32_e32 v1, 16, v1
-; GFX8-NEXT: v_alignbit_b32 v2, v2, v19, 16
-; GFX8-NEXT: v_alignbit_b32 v3, v3, v20, 16
-; GFX8-NEXT: v_alignbit_b32 v4, v4, v21, 16
-; GFX8-NEXT: v_alignbit_b32 v5, v5, v22, 16
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v6
+; GFX8-NEXT: v_lshrrev_b32_e32 v7, 16, v7
+; GFX8-NEXT: v_lshrrev_b32_e32 v8, 16, v8
+; GFX8-NEXT: v_lshrrev_b32_e32 v9, 16, v9
+; GFX8-NEXT: v_lshrrev_b32_e32 v10, 16, v10
; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v17
; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v18
-; GFX8-NEXT: v_lshrrev_b32_e32 v19, 16, v30
-; GFX8-NEXT: v_lshrrev_b32_e32 v20, 16, v29
-; GFX8-NEXT: v_lshrrev_b32_e32 v21, 16, v28
-; GFX8-NEXT: v_lshrrev_b32_e32 v22, 16, v27
-; GFX8-NEXT: v_lshrrev_b32_e32 v9, 16, v9
-; GFX8-NEXT: v_lshrrev_b32_e32 v8, 16, v8
-; GFX8-NEXT: v_lshrrev_b32_e32 v7, 16, v7
+; GFX8-NEXT: v_lshrrev_b32_e32 v19, 16, v19
+; GFX8-NEXT: v_lshrrev_b32_e32 v12, 16, v12
+; GFX8-NEXT: v_lshrrev_b32_e32 v11, 16, v11
; GFX8-NEXT: v_alignbit_b32 v0, v0, v31, 16
-; GFX8-NEXT: v_alignbit_b32 v1, v1, v26, 16
+; GFX8-NEXT: v_alignbit_b32 v1, v1, v30, 16
+; GFX8-NEXT: v_alignbit_b32 v2, v2, v29, 16
+; GFX8-NEXT: v_alignbit_b32 v3, v3, v20, 16
+; GFX8-NEXT: v_alignbit_b32 v4, v4, v21, 16
+; GFX8-NEXT: v_alignbit_b32 v5, v5, v22, 16
; GFX8-NEXT: v_alignbit_b32 v6, v6, v23, 16
; GFX8-NEXT: v_alignbit_b32 v7, v7, v24, 16
; GFX8-NEXT: v_alignbit_b32 v8, v8, v25, 16
-; GFX8-NEXT: v_alignbit_b32 v9, v9, v10, 16
-; GFX8-NEXT: v_alignbit_b32 v10, v22, v11, 16
-; GFX8-NEXT: v_alignbit_b32 v11, v21, v12, 16
-; GFX8-NEXT: v_alignbit_b32 v12, v20, v13, 16
+; GFX8-NEXT: v_alignbit_b32 v9, v9, v26, 16
+; GFX8-NEXT: v_alignbit_b32 v10, v10, v27, 16
+; GFX8-NEXT: v_alignbit_b32 v11, v11, v28, 16
+; GFX8-NEXT: v_alignbit_b32 v12, v12, v13, 16
; GFX8-NEXT: v_alignbit_b32 v13, v19, v14, 16
; GFX8-NEXT: v_alignbit_b32 v14, v18, v15, 16
; GFX8-NEXT: v_alignbit_b32 v15, v17, v16, 16
diff --git a/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs-packed.ll b/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs-packed.ll
index 03525a359de91..cba31b03e6025 100644
--- a/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs-packed.ll
+++ b/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs-packed.ll
@@ -683,7 +683,6 @@ define void @too_many_args_use_workitem_id_x(
; GFX7-NEXT: v_and_b32_e32 v31, 0x3ff, v31
; GFX7-NEXT: flat_store_dword v[0:1], v31
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-NEXT: flat_store_dword v[0:1], v0
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v1
@@ -692,6 +691,8 @@ define void @too_many_args_use_workitem_id_x(
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v3
; GFX7-NEXT: s_waitcnt vmcnt(0)
+; GFX7-NEXT: buffer_load_dword v0, off, s[0:3], s32
+; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v4
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v5
@@ -746,7 +747,7 @@ define void @too_many_args_use_workitem_id_x(
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v30
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: flat_store_dword v[0:1], v31
+; GFX7-NEXT: flat_store_dword v[0:1], v0
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
@@ -1124,7 +1125,6 @@ define void @too_many_args_use_workitem_id_x_byval(
; GFX7-NEXT: v_and_b32_e32 v31, 0x3ff, v31
; GFX7-NEXT: flat_store_dword v[0:1], v31
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-NEXT: flat_store_dword v[0:1], v0
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v1
@@ -1133,6 +1133,8 @@ define void @too_many_args_use_workitem_id_x_byval(
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v3
; GFX7-NEXT: s_waitcnt vmcnt(0)
+; GFX7-NEXT: buffer_load_dword v0, off, s[0:3], s32
+; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v4
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v5
@@ -1187,7 +1189,7 @@ define void @too_many_args_use_workitem_id_x_byval(
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: flat_store_dword v[0:1], v30
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: flat_store_dword v[0:1], v31
+; GFX7-NEXT: flat_store_dword v[0:1], v0
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 glc
; GFX7-NEXT: s_waitcnt vmcnt(0)
diff --git a/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs.ll b/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs.ll
index d17327e66bade..a7c6846424450 100644
--- a/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs.ll
+++ b/llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs.ll
@@ -469,7 +469,6 @@ define void @too_many_args_use_workitem_id_x(
; GCN-NEXT: v_and_b32_e32 v31, 0x3ff, v31
; GCN-NEXT: flat_store_dword v[0:1], v31
; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GCN-NEXT: flat_store_dword v[0:1], v0
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v1
@@ -478,6 +477,8 @@ define void @too_many_args_use_workitem_id_x(
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v3
; GCN-NEXT: s_waitcnt vmcnt(0)
+; GCN-NEXT: buffer_load_dword v0, off, s[0:3], s32
+; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v4
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v5
@@ -532,7 +533,7 @@ define void @too_many_args_use_workitem_id_x(
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v30
; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: flat_store_dword v[0:1], v31
+; GCN-NEXT: flat_store_dword v[0:1], v0
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: s_setpc_b64 s[30:31]
i32 %arg0, i32 %arg1, i32 %arg2, i32 %arg3, i32 %arg4, i32 %arg5, i32 %arg6, i32 %arg7,
@@ -774,7 +775,6 @@ define void @too_many_args_use_workitem_id_x_byval(
; GCN-NEXT: v_and_b32_e32 v31, 0x3ff, v31
; GCN-NEXT: flat_store_dword v[0:1], v31
; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GCN-NEXT: flat_store_dword v[0:1], v0
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v1
@@ -783,6 +783,8 @@ define void @too_many_args_use_workitem_id_x_byval(
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v3
; GCN-NEXT: s_waitcnt vmcnt(0)
+; GCN-NEXT: buffer_load_dword v0, off, s[0:3], s32
+; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v4
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v5
@@ -837,7 +839,7 @@ define void @too_many_args_use_workitem_id_x_byval(
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: flat_store_dword v[0:1], v30
; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: flat_store_dword v[0:1], v31
+; GCN-NEXT: flat_store_dword v[0:1], v0
; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 glc
; GCN-NEXT: s_waitcnt vmcnt(0)
diff --git a/llvm/test/CodeGen/AMDGPU/debug-value-scheduler-crash.mir b/llvm/test/CodeGen/AMDGPU/debug-value-scheduler-crash.mir
index 1120d125a05e4..07b0f9ed75d17 100644
--- a/llvm/test/CodeGen/AMDGPU/debug-value-scheduler-crash.mir
+++ b/llvm/test/CodeGen/AMDGPU/debug-value-scheduler-crash.mir
@@ -25,7 +25,7 @@ body: |
; CHECK: bb.0:
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[DEF:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
+ ; CHECK-NEXT: [[DEF:%[0-9]+]]:vreg_64 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF1:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF2:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF3:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
@@ -33,9 +33,10 @@ body: |
; CHECK-NEXT: [[DEF5:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF6:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF7:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
- ; CHECK-NEXT: [[V_MUL_F32_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 1082130432, [[DEF]], implicit $mode, implicit $exec
; CHECK-NEXT: [[DEF8:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
+ ; CHECK-NEXT: [[V_MUL_F32_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 1082130432, [[DEF1]], implicit $mode, implicit $exec
; CHECK-NEXT: [[DEF9:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
+ ; CHECK-NEXT: [[DEF10:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.2(0x80000000)
@@ -51,34 +52,33 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
; CHECK-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
- ; CHECK-NEXT: [[DEF10:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF11:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
+ ; CHECK-NEXT: [[DEF12:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY [[V_MOV_B32_e32_]]
- ; CHECK-NEXT: [[V_MUL_F32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[DEF6]], [[DEF6]], implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MUL_F32_e32_2:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[DEF6]], [[DEF6]], implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MUL_F32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[DEF7]], [[DEF7]], implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MUL_F32_e32_2:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[DEF7]], [[DEF7]], implicit $mode, implicit $exec
; CHECK-NEXT: [[V_MUL_F32_e32_3:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[V_MOV_B32_e32_]], [[V_MOV_B32_e32_]], implicit $mode, implicit $exec
; CHECK-NEXT: [[V_MOV_B32_e32_1:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 1092616192, implicit $exec
- ; CHECK-NEXT: [[DEF12:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
+ ; CHECK-NEXT: [[DEF13:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: [[V_ADD_F32_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_ADD_F32_e32 [[V_MOV_B32_e32_]], [[V_MOV_B32_e32_]], implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MUL_F32_e32_4:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[DEF6]], [[DEF6]], implicit $mode, implicit $exec
- ; CHECK-NEXT: dead [[V_MUL_F32_e32_5:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[V_MUL_F32_e32_4]], [[DEF12]], implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MUL_F32_e32_4:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[DEF7]], [[DEF7]], implicit $mode, implicit $exec
+ ; CHECK-NEXT: dead [[V_MUL_F32_e32_5:%[0-9]+]]:vgpr_32 = nofpexcept V_MUL_F32_e32 [[V_MUL_F32_e32_4]], [[DEF13]], implicit $mode, implicit $exec
; CHECK-NEXT: dead [[V_MOV_B32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MAC_F32_e32 [[V_ADD_F32_e32_]], [[COPY]], [[V_MOV_B32_e32_1]], implicit $mode, implicit $exec
- ; CHECK-NEXT: [[DEF13:%[0-9]+]]:sreg_64 = IMPLICIT_DEF
+ ; CHECK-NEXT: [[DEF14:%[0-9]+]]:sreg_64 = IMPLICIT_DEF
; CHECK-NEXT: $sgpr4 = IMPLICIT_DEF
- ; CHECK-NEXT: $vgpr0 = COPY [[DEF10]]
+ ; CHECK-NEXT: $vgpr0 = COPY [[DEF11]]
; CHECK-NEXT: $vgpr0 = COPY [[V_MOV_B32_e32_]]
- ; CHECK-NEXT: $vgpr1 = COPY [[DEF6]]
+ ; CHECK-NEXT: $vgpr1 = COPY [[DEF7]]
; CHECK-NEXT: $vgpr0 = COPY [[V_MUL_F32_e32_1]]
; CHECK-NEXT: $vgpr1 = COPY [[V_MUL_F32_e32_2]]
; CHECK-NEXT: $vgpr2 = COPY [[V_MUL_F32_e32_3]]
- ; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL [[DEF13]], @foo, csr_amdgpu, implicit undef $sgpr0_sgpr1_sgpr2_sgpr3, implicit killed $sgpr4, implicit $vgpr0, implicit $vgpr1, implicit $vgpr2, implicit-def $vgpr0
- ; CHECK-NEXT: [[V_ADD_F32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_ADD_F32_e32 [[V_MUL_F32_e32_]], [[DEF7]], implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_ADD_F32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MAC_F32_e32 [[DEF11]], [[DEF8]], [[V_ADD_F32_e32_1]], implicit $mode, implicit $exec
- ; CHECK-NEXT: dead [[V_MAD_F32_e64_:%[0-9]+]]:vgpr_32 = nofpexcept V_MAD_F32_e64 0, [[V_ADD_F32_e32_1]], 0, [[DEF3]], 0, [[DEF]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: dead [[V_MAD_F32_e64_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MAD_F32_e64 0, [[V_ADD_F32_e32_1]], 0, [[DEF4]], 0, [[DEF1]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: dead [[V_MAD_F32_e64_2:%[0-9]+]]:vgpr_32 = nofpexcept V_MAD_F32_e64 0, [[V_ADD_F32_e32_1]], 0, [[DEF5]], 0, [[DEF2]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[DEF14:%[0-9]+]]:vreg_64 = IMPLICIT_DEF
- ; CHECK-NEXT: GLOBAL_STORE_DWORD [[DEF14]], [[DEF9]], 0, 0, implicit $exec
+ ; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL [[DEF14]], @foo, csr_amdgpu, implicit undef $sgpr0_sgpr1_sgpr2_sgpr3, implicit killed $sgpr4, implicit $vgpr0, implicit $vgpr1, implicit $vgpr2, implicit-def $vgpr0
+ ; CHECK-NEXT: [[V_ADD_F32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_ADD_F32_e32 [[V_MUL_F32_e32_]], [[DEF8]], implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_ADD_F32_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MAC_F32_e32 [[DEF12]], [[DEF9]], [[V_ADD_F32_e32_1]], implicit $mode, implicit $exec
+ ; CHECK-NEXT: dead [[V_MAD_F32_e64_:%[0-9]+]]:vgpr_32 = nofpexcept V_MAD_F32_e64 0, [[V_ADD_F32_e32_1]], 0, [[DEF4]], 0, [[DEF1]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: dead [[V_MAD_F32_e64_1:%[0-9]+]]:vgpr_32 = nofpexcept V_MAD_F32_e64 0, [[V_ADD_F32_e32_1]], 0, [[DEF5]], 0, [[DEF2]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: dead [[V_MAD_F32_e64_2:%[0-9]+]]:vgpr_32 = nofpexcept V_MAD_F32_e64 0, [[V_ADD_F32_e32_1]], 0, [[DEF6]], 0, [[DEF3]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: GLOBAL_STORE_DWORD [[DEF]], [[DEF10]], 0, 0, implicit $exec
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1
diff --git a/llvm/test/CodeGen/AMDGPU/div_i128.ll b/llvm/test/CodeGen/AMDGPU/div_i128.ll
index 5ff48c8c20c3e..7606d5e3bc404 100644
--- a/llvm/test/CodeGen/AMDGPU/div_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/div_i128.ll
@@ -9,26 +9,26 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-LABEL: v_sdiv_i128_vv:
; GFX9: ; %bb.0: ; %_udiv-special-cases
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT: v_ashrrev_i32_e32 v14, 31, v3
-; GFX9-NEXT: v_xor_b32_e32 v0, v0, v14
-; GFX9-NEXT: v_xor_b32_e32 v1, v1, v14
-; GFX9-NEXT: v_sub_co_u32_e32 v8, vcc, v0, v14
-; GFX9-NEXT: v_subb_co_u32_e32 v9, vcc, v1, v14, vcc
-; GFX9-NEXT: v_xor_b32_e32 v1, v2, v14
-; GFX9-NEXT: v_ashrrev_i32_e32 v15, 31, v7
-; GFX9-NEXT: v_xor_b32_e32 v0, v3, v14
-; GFX9-NEXT: v_subb_co_u32_e32 v10, vcc, v1, v14, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v11, vcc, v0, v14, vcc
-; GFX9-NEXT: v_xor_b32_e32 v1, v4, v15
-; GFX9-NEXT: v_xor_b32_e32 v0, v5, v15
-; GFX9-NEXT: v_sub_co_u32_e32 v16, vcc, v1, v15
-; GFX9-NEXT: v_subb_co_u32_e32 v17, vcc, v0, v15, vcc
-; GFX9-NEXT: v_xor_b32_e32 v0, v6, v15
-; GFX9-NEXT: v_xor_b32_e32 v1, v7, v15
-; GFX9-NEXT: v_subb_co_u32_e32 v0, vcc, v0, v15, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v15, vcc
-; GFX9-NEXT: v_or_b32_e32 v3, v17, v1
-; GFX9-NEXT: v_or_b32_e32 v2, v16, v0
+; GFX9-NEXT: v_ashrrev_i32_e32 v16, 31, v3
+; GFX9-NEXT: v_xor_b32_e32 v0, v0, v16
+; GFX9-NEXT: v_xor_b32_e32 v1, v1, v16
+; GFX9-NEXT: v_sub_co_u32_e32 v8, vcc, v0, v16
+; GFX9-NEXT: v_subb_co_u32_e32 v9, vcc, v1, v16, vcc
+; GFX9-NEXT: v_xor_b32_e32 v1, v2, v16
+; GFX9-NEXT: v_ashrrev_i32_e32 v17, 31, v7
+; GFX9-NEXT: v_xor_b32_e32 v0, v3, v16
+; GFX9-NEXT: v_subb_co_u32_e32 v10, vcc, v1, v16, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v11, vcc, v0, v16, vcc
+; GFX9-NEXT: v_xor_b32_e32 v1, v4, v17
+; GFX9-NEXT: v_xor_b32_e32 v0, v5, v17
+; GFX9-NEXT: v_sub_co_u32_e32 v18, vcc, v1, v17
+; GFX9-NEXT: v_subb_co_u32_e32 v19, vcc, v0, v17, vcc
+; GFX9-NEXT: v_xor_b32_e32 v0, v6, v17
+; GFX9-NEXT: v_xor_b32_e32 v1, v7, v17
+; GFX9-NEXT: v_subb_co_u32_e32 v0, vcc, v0, v17, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v17, vcc
+; GFX9-NEXT: v_or_b32_e32 v3, v19, v1
+; GFX9-NEXT: v_or_b32_e32 v2, v18, v0
; GFX9-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[2:3]
; GFX9-NEXT: v_or_b32_e32 v3, v9, v11
; GFX9-NEXT: v_or_b32_e32 v2, v8, v10
@@ -37,9 +37,9 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: v_add_u32_e32 v2, 32, v2
; GFX9-NEXT: v_ffbh_u32_e32 v3, v1
; GFX9-NEXT: v_min_u32_e32 v2, v2, v3
-; GFX9-NEXT: v_ffbh_u32_e32 v3, v16
+; GFX9-NEXT: v_ffbh_u32_e32 v3, v18
; GFX9-NEXT: v_add_u32_e32 v3, 32, v3
-; GFX9-NEXT: v_ffbh_u32_e32 v4, v17
+; GFX9-NEXT: v_ffbh_u32_e32 v4, v19
; GFX9-NEXT: v_min_u32_e32 v3, v3, v4
; GFX9-NEXT: s_or_b64 s[4:5], vcc, s[4:5]
; GFX9-NEXT: v_add_co_u32_e32 v3, vcc, 64, v3
@@ -87,11 +87,11 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: s_and_saveexec_b64 s[6:7], s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB0_6
; GFX9-NEXT: ; %bb.1: ; %udiv-bb1
-; GFX9-NEXT: v_add_co_u32_e32 v18, vcc, 1, v2
-; GFX9-NEXT: v_addc_co_u32_e32 v19, vcc, 0, v3, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v20, vcc, 0, v4, vcc
+; GFX9-NEXT: v_add_co_u32_e32 v20, vcc, 1, v2
+; GFX9-NEXT: v_addc_co_u32_e32 v21, vcc, 0, v3, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v22, vcc, 0, v4, vcc
; GFX9-NEXT: v_sub_u32_e32 v7, 0x7f, v2
-; GFX9-NEXT: v_addc_co_u32_e32 v21, vcc, 0, v5, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v23, vcc, 0, v5, vcc
; GFX9-NEXT: v_sub_u32_e32 v5, 64, v7
; GFX9-NEXT: v_lshlrev_b64 v[3:4], v7, v[10:11]
; GFX9-NEXT: v_lshrrev_b64 v[5:6], v5, v[8:9]
@@ -115,71 +115,71 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: s_xor_b64 s[8:9], exec, s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB0_5
; GFX9-NEXT: ; %bb.2: ; %udiv-preheader
-; GFX9-NEXT: v_sub_u32_e32 v12, 64, v18
-; GFX9-NEXT: v_lshrrev_b64 v[6:7], v18, v[8:9]
+; GFX9-NEXT: v_sub_u32_e32 v12, 64, v20
+; GFX9-NEXT: v_lshrrev_b64 v[6:7], v20, v[8:9]
; GFX9-NEXT: v_lshlrev_b64 v[12:13], v12, v[10:11]
-; GFX9-NEXT: v_cmp_gt_u32_e32 vcc, 64, v18
+; GFX9-NEXT: v_cmp_gt_u32_e32 vcc, 64, v20
; GFX9-NEXT: v_or_b32_e32 v12, v6, v12
-; GFX9-NEXT: v_subrev_u32_e32 v6, 64, v18
+; GFX9-NEXT: v_subrev_u32_e32 v6, 64, v20
; GFX9-NEXT: v_or_b32_e32 v13, v7, v13
; GFX9-NEXT: v_lshrrev_b64 v[6:7], v6, v[10:11]
-; GFX9-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v18
+; GFX9-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v20
; GFX9-NEXT: v_cndmask_b32_e32 v7, v7, v13, vcc
; GFX9-NEXT: v_cndmask_b32_e64 v9, v7, v9, s[4:5]
; GFX9-NEXT: v_cndmask_b32_e32 v12, v6, v12, vcc
-; GFX9-NEXT: v_lshrrev_b64 v[6:7], v18, v[10:11]
+; GFX9-NEXT: v_lshrrev_b64 v[6:7], v20, v[10:11]
; GFX9-NEXT: v_cndmask_b32_e64 v8, v12, v8, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e32 v11, 0, v7, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v10, 0, v6, vcc
-; GFX9-NEXT: v_add_co_u32_e32 v22, vcc, -1, v16
-; GFX9-NEXT: v_addc_co_u32_e32 v23, vcc, -1, v17, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v24, vcc, -1, v0, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v25, vcc, -1, v1, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v13, 0, v7, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v12, 0, v6, vcc
+; GFX9-NEXT: v_add_co_u32_e32 v24, vcc, -1, v18
+; GFX9-NEXT: v_addc_co_u32_e32 v25, vcc, -1, v19, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v26, vcc, -1, v0, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v27, vcc, -1, v1, vcc
+; GFX9-NEXT: v_mov_b32_e32 v10, 0
+; GFX9-NEXT: v_mov_b32_e32 v11, 0
; GFX9-NEXT: s_mov_b64 s[4:5], 0
-; GFX9-NEXT: v_mov_b32_e32 v12, 0
-; GFX9-NEXT: v_mov_b32_e32 v13, 0
+; GFX9-NEXT: v_mov_b32_e32 v14, 0
+; GFX9-NEXT: v_mov_b32_e32 v15, 0
; GFX9-NEXT: v_mov_b32_e32 v7, 0
; GFX9-NEXT: .LBB0_3: ; %udiv-do-while
; GFX9-NEXT: ; =>This Inner Loop Header: Depth=1
; GFX9-NEXT: v_lshrrev_b32_e32 v6, 31, v5
; GFX9-NEXT: v_lshlrev_b64 v[4:5], 1, v[4:5]
-; GFX9-NEXT: v_lshrrev_b32_e32 v26, 31, v3
-; GFX9-NEXT: v_lshlrev_b64 v[2:3], 1, v[2:3]
-; GFX9-NEXT: v_or_b32_e32 v4, v12, v4
-; GFX9-NEXT: v_mov_b32_e32 v12, 0
-; GFX9-NEXT: v_or3_b32 v2, v2, v6, v12
-; GFX9-NEXT: v_lshrrev_b32_e32 v6, 31, v9
+; GFX9-NEXT: v_lshlrev_b64 v[12:13], 1, v[12:13]
+; GFX9-NEXT: v_or_b32_e32 v4, v14, v4
+; GFX9-NEXT: v_lshrrev_b32_e32 v14, 31, v9
; GFX9-NEXT: v_lshlrev_b64 v[8:9], 1, v[8:9]
-; GFX9-NEXT: v_lshlrev_b64 v[10:11], 1, v[10:11]
-; GFX9-NEXT: v_or_b32_e32 v8, v8, v26
-; GFX9-NEXT: v_or_b32_e32 v6, v10, v6
-; GFX9-NEXT: v_sub_co_u32_e32 v10, vcc, v22, v8
-; GFX9-NEXT: v_subb_co_u32_e32 v10, vcc, v23, v9, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v10, vcc, v24, v6, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v10, vcc, v25, v11, vcc
-; GFX9-NEXT: v_ashrrev_i32_e32 v26, 31, v10
-; GFX9-NEXT: v_and_b32_e32 v10, v26, v16
-; GFX9-NEXT: v_and_b32_e32 v12, v26, v17
-; GFX9-NEXT: v_sub_co_u32_e32 v8, vcc, v8, v10
-; GFX9-NEXT: v_subb_co_u32_e32 v9, vcc, v9, v12, vcc
-; GFX9-NEXT: v_and_b32_e32 v10, v26, v0
-; GFX9-NEXT: v_and_b32_e32 v12, v26, v1
-; GFX9-NEXT: v_subb_co_u32_e32 v10, vcc, v6, v10, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v11, vcc, v11, v12, vcc
-; GFX9-NEXT: v_add_co_u32_e32 v18, vcc, -1, v18
-; GFX9-NEXT: v_addc_co_u32_e32 v19, vcc, -1, v19, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v20, vcc, -1, v20, vcc
-; GFX9-NEXT: v_or_b32_e32 v5, v13, v5
-; GFX9-NEXT: v_mov_b32_e32 v13, 0
+; GFX9-NEXT: v_or_b32_e32 v12, v12, v14
+; GFX9-NEXT: v_lshrrev_b32_e32 v14, 31, v3
+; GFX9-NEXT: v_or_b32_e32 v8, v8, v14
+; GFX9-NEXT: v_sub_co_u32_e32 v14, vcc, v24, v8
+; GFX9-NEXT: v_subb_co_u32_e32 v14, vcc, v25, v9, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v14, vcc, v26, v12, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v14, vcc, v27, v13, vcc
+; GFX9-NEXT: v_ashrrev_i32_e32 v28, 31, v14
+; GFX9-NEXT: v_and_b32_e32 v14, v28, v18
+; GFX9-NEXT: v_sub_co_u32_e32 v8, vcc, v8, v14
+; GFX9-NEXT: v_and_b32_e32 v14, v28, v19
+; GFX9-NEXT: v_subb_co_u32_e32 v9, vcc, v9, v14, vcc
+; GFX9-NEXT: v_and_b32_e32 v14, v28, v0
+; GFX9-NEXT: v_subb_co_u32_e32 v12, vcc, v12, v14, vcc
+; GFX9-NEXT: v_and_b32_e32 v14, v28, v1
+; GFX9-NEXT: v_subb_co_u32_e32 v13, vcc, v13, v14, vcc
+; GFX9-NEXT: v_add_co_u32_e32 v20, vcc, -1, v20
; GFX9-NEXT: v_addc_co_u32_e32 v21, vcc, -1, v21, vcc
-; GFX9-NEXT: v_or3_b32 v3, v3, 0, v13
-; GFX9-NEXT: v_or_b32_e32 v12, v18, v20
-; GFX9-NEXT: v_or_b32_e32 v13, v19, v21
-; GFX9-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[12:13]
-; GFX9-NEXT: v_and_b32_e32 v6, 1, v26
+; GFX9-NEXT: v_addc_co_u32_e32 v22, vcc, -1, v22, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v23, vcc, -1, v23, vcc
+; GFX9-NEXT: v_or_b32_e32 v5, v15, v5
+; GFX9-NEXT: v_lshlrev_b64 v[2:3], 1, v[2:3]
+; GFX9-NEXT: v_or_b32_e32 v14, v20, v22
+; GFX9-NEXT: v_or_b32_e32 v15, v21, v23
+; GFX9-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[14:15]
+; GFX9-NEXT: v_or3_b32 v2, v2, v6, v10
+; GFX9-NEXT: v_and_b32_e32 v6, 1, v28
+; GFX9-NEXT: v_or3_b32 v3, v3, 0, v11
; GFX9-NEXT: s_or_b64 s[4:5], vcc, s[4:5]
-; GFX9-NEXT: v_mov_b32_e32 v13, v7
-; GFX9-NEXT: v_mov_b32_e32 v12, v6
+; GFX9-NEXT: v_mov_b32_e32 v15, v7
+; GFX9-NEXT: v_mov_b32_e32 v14, v6
; GFX9-NEXT: s_andn2_b64 exec, exec, s[4:5]
; GFX9-NEXT: s_cbranch_execnz .LBB0_3
; GFX9-NEXT: ; %bb.4: ; %Flow
@@ -194,7 +194,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: v_or_b32_e32 v6, v6, v0
; GFX9-NEXT: .LBB0_6: ; %Flow3
; GFX9-NEXT: s_or_b64 exec, exec, s[6:7]
-; GFX9-NEXT: v_xor_b32_e32 v3, v15, v14
+; GFX9-NEXT: v_xor_b32_e32 v3, v17, v16
; GFX9-NEXT: v_xor_b32_e32 v0, v6, v3
; GFX9-NEXT: v_xor_b32_e32 v1, v7, v3
; GFX9-NEXT: v_sub_co_u32_e32 v0, vcc, v0, v3
diff --git a/llvm/test/CodeGen/AMDGPU/div_v2i128.ll b/llvm/test/CodeGen/AMDGPU/div_v2i128.ll
index 6789df5453d16..c9cc34f9e8bc1 100644
--- a/llvm/test/CodeGen/AMDGPU/div_v2i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/div_v2i128.ll
@@ -6,26 +6,26 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-LABEL: v_sdiv_v2i128_vv:
; SDAG: ; %bb.0: ; %_udiv-special-cases_udiv-special-cases
; SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SDAG-NEXT: v_ashrrev_i32_e32 v25, 31, v3
-; SDAG-NEXT: v_xor_b32_e32 v0, v0, v25
-; SDAG-NEXT: v_xor_b32_e32 v1, v1, v25
-; SDAG-NEXT: v_sub_i32_e32 v16, vcc, v0, v25
-; SDAG-NEXT: v_subb_u32_e32 v17, vcc, v1, v25, vcc
-; SDAG-NEXT: v_xor_b32_e32 v1, v2, v25
-; SDAG-NEXT: v_ashrrev_i32_e32 v26, 31, v11
-; SDAG-NEXT: v_xor_b32_e32 v0, v3, v25
-; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v1, v25, vcc
-; SDAG-NEXT: v_subb_u32_e32 v19, vcc, v0, v25, vcc
-; SDAG-NEXT: v_xor_b32_e32 v1, v8, v26
-; SDAG-NEXT: v_xor_b32_e32 v0, v9, v26
-; SDAG-NEXT: v_sub_i32_e32 v27, vcc, v1, v26
-; SDAG-NEXT: v_subb_u32_e32 v28, vcc, v0, v26, vcc
-; SDAG-NEXT: v_xor_b32_e32 v1, v10, v26
-; SDAG-NEXT: v_xor_b32_e32 v0, v11, v26
-; SDAG-NEXT: v_subb_u32_e32 v2, vcc, v1, v26, vcc
-; SDAG-NEXT: v_subb_u32_e32 v3, vcc, v0, v26, vcc
-; SDAG-NEXT: v_or_b32_e32 v1, v28, v3
-; SDAG-NEXT: v_or_b32_e32 v0, v27, v2
+; SDAG-NEXT: v_ashrrev_i32_e32 v22, 31, v3
+; SDAG-NEXT: v_xor_b32_e32 v0, v0, v22
+; SDAG-NEXT: v_xor_b32_e32 v1, v1, v22
+; SDAG-NEXT: v_sub_i32_e32 v16, vcc, v0, v22
+; SDAG-NEXT: v_subb_u32_e32 v17, vcc, v1, v22, vcc
+; SDAG-NEXT: v_xor_b32_e32 v1, v2, v22
+; SDAG-NEXT: v_ashrrev_i32_e32 v23, 31, v11
+; SDAG-NEXT: v_xor_b32_e32 v0, v3, v22
+; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v1, v22, vcc
+; SDAG-NEXT: v_subb_u32_e32 v19, vcc, v0, v22, vcc
+; SDAG-NEXT: v_xor_b32_e32 v1, v8, v23
+; SDAG-NEXT: v_xor_b32_e32 v0, v9, v23
+; SDAG-NEXT: v_sub_i32_e32 v24, vcc, v1, v23
+; SDAG-NEXT: v_subb_u32_e32 v25, vcc, v0, v23, vcc
+; SDAG-NEXT: v_xor_b32_e32 v1, v10, v23
+; SDAG-NEXT: v_xor_b32_e32 v0, v11, v23
+; SDAG-NEXT: v_subb_u32_e32 v2, vcc, v1, v23, vcc
+; SDAG-NEXT: v_subb_u32_e32 v3, vcc, v0, v23, vcc
+; SDAG-NEXT: v_or_b32_e32 v1, v25, v3
+; SDAG-NEXT: v_or_b32_e32 v0, v24, v2
; SDAG-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[0:1]
; SDAG-NEXT: v_or_b32_e32 v1, v17, v19
; SDAG-NEXT: v_or_b32_e32 v0, v16, v18
@@ -34,9 +34,9 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_add_i32_e64 v0, s[6:7], 32, v0
; SDAG-NEXT: v_ffbh_u32_e32 v1, v3
; SDAG-NEXT: v_min_u32_e32 v0, v0, v1
-; SDAG-NEXT: v_ffbh_u32_e32 v1, v27
+; SDAG-NEXT: v_ffbh_u32_e32 v1, v24
; SDAG-NEXT: v_add_i32_e64 v1, s[6:7], 32, v1
-; SDAG-NEXT: v_ffbh_u32_e32 v8, v28
+; SDAG-NEXT: v_ffbh_u32_e32 v8, v25
; SDAG-NEXT: v_min_u32_e32 v1, v1, v8
; SDAG-NEXT: v_add_i32_e64 v1, s[6:7], 64, v1
; SDAG-NEXT: v_addc_u32_e64 v8, s[6:7], 0, 0, s[6:7]
@@ -85,10 +85,10 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: s_and_saveexec_b64 s[6:7], s[10:11]
; SDAG-NEXT: s_cbranch_execz .LBB0_6
; SDAG-NEXT: ; %bb.1: ; %udiv-bb15
-; SDAG-NEXT: v_add_i32_e32 v29, vcc, 1, v8
-; SDAG-NEXT: v_addc_u32_e32 v30, vcc, 0, v9, vcc
-; SDAG-NEXT: v_addc_u32_e32 v31, vcc, 0, v10, vcc
-; SDAG-NEXT: v_addc_u32_e64 v32, s[4:5], 0, v11, vcc
+; SDAG-NEXT: v_add_i32_e32 v26, vcc, 1, v8
+; SDAG-NEXT: v_addc_u32_e32 v27, vcc, 0, v9, vcc
+; SDAG-NEXT: v_addc_u32_e32 v28, vcc, 0, v10, vcc
+; SDAG-NEXT: v_addc_u32_e64 v29, s[4:5], 0, v11, vcc
; SDAG-NEXT: v_sub_i32_e32 v11, vcc, s8, v8
; SDAG-NEXT: v_sub_i32_e32 v9, vcc, 64, v11
; SDAG-NEXT: v_lshl_b64 v[0:1], v[18:19], v11
@@ -113,72 +113,72 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: s_xor_b64 s[8:9], exec, s[4:5]
; SDAG-NEXT: s_cbranch_execz .LBB0_5
; SDAG-NEXT: ; %bb.2: ; %udiv-preheader4
-; SDAG-NEXT: v_sub_i32_e32 v20, vcc, 64, v29
-; SDAG-NEXT: v_lshr_b64 v[10:11], v[16:17], v29
+; SDAG-NEXT: v_sub_i32_e32 v20, vcc, 64, v26
+; SDAG-NEXT: v_lshr_b64 v[10:11], v[16:17], v26
; SDAG-NEXT: v_lshl_b64 v[20:21], v[18:19], v20
-; SDAG-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v29
-; SDAG-NEXT: v_or_b32_e32 v22, v10, v20
-; SDAG-NEXT: v_subrev_i32_e32 v10, vcc, 64, v29
+; SDAG-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v26
+; SDAG-NEXT: v_or_b32_e32 v20, v10, v20
+; SDAG-NEXT: v_subrev_i32_e32 v10, vcc, 64, v26
; SDAG-NEXT: v_or_b32_e32 v21, v11, v21
; SDAG-NEXT: v_lshr_b64 v[10:11], v[18:19], v10
-; SDAG-NEXT: v_cmp_gt_u32_e32 vcc, 64, v29
+; SDAG-NEXT: v_cmp_gt_u32_e32 vcc, 64, v26
; SDAG-NEXT: v_cndmask_b32_e32 v11, v11, v21, vcc
-; SDAG-NEXT: v_cndmask_b32_e64 v20, v11, v17, s[4:5]
-; SDAG-NEXT: v_cndmask_b32_e32 v17, v10, v22, vcc
-; SDAG-NEXT: v_lshr_b64 v[10:11], v[18:19], v29
-; SDAG-NEXT: v_cndmask_b32_e64 v19, v17, v16, s[4:5]
-; SDAG-NEXT: v_cndmask_b32_e32 v22, 0, v11, vcc
-; SDAG-NEXT: v_cndmask_b32_e32 v21, 0, v10, vcc
-; SDAG-NEXT: v_add_i32_e32 v18, vcc, -1, v27
-; SDAG-NEXT: v_addc_u32_e32 v33, vcc, -1, v28, vcc
-; SDAG-NEXT: v_addc_u32_e32 v34, vcc, -1, v2, vcc
-; SDAG-NEXT: v_addc_u32_e32 v35, vcc, -1, v3, vcc
-; SDAG-NEXT: v_mov_b32_e32 v16, 0
-; SDAG-NEXT: v_mov_b32_e32 v17, 0
+; SDAG-NEXT: v_cndmask_b32_e64 v17, v11, v17, s[4:5]
+; SDAG-NEXT: v_cndmask_b32_e32 v20, v10, v20, vcc
+; SDAG-NEXT: v_lshr_b64 v[10:11], v[18:19], v26
+; SDAG-NEXT: v_cndmask_b32_e64 v16, v20, v16, s[4:5]
+; SDAG-NEXT: v_cndmask_b32_e32 v19, 0, v11, vcc
+; SDAG-NEXT: v_cndmask_b32_e32 v18, 0, v10, vcc
+; SDAG-NEXT: v_add_i32_e32 v30, vcc, -1, v24
+; SDAG-NEXT: v_addc_u32_e32 v31, vcc, -1, v25, vcc
+; SDAG-NEXT: v_addc_u32_e32 v32, vcc, -1, v2, vcc
+; SDAG-NEXT: v_addc_u32_e32 v33, vcc, -1, v3, vcc
; SDAG-NEXT: s_mov_b64 s[4:5], 0
-; SDAG-NEXT: v_mov_b32_e32 v23, 0
-; SDAG-NEXT: v_mov_b32_e32 v24, 0
+; SDAG-NEXT: v_mov_b32_e32 v20, 0
+; SDAG-NEXT: v_mov_b32_e32 v21, 0
; SDAG-NEXT: v_mov_b32_e32 v11, 0
; SDAG-NEXT: .LBB0_3: ; %udiv-do-while3
; SDAG-NEXT: ; =>This Inner Loop Header: Depth=1
-; SDAG-NEXT: v_lshl_b64 v[36:37], v[8:9], 1
-; SDAG-NEXT: v_lshl_b64 v[21:22], v[21:22], 1
-; SDAG-NEXT: v_or_b32_e32 v8, v23, v36
-; SDAG-NEXT: v_lshrrev_b32_e32 v23, 31, v20
-; SDAG-NEXT: v_lshl_b64 v[19:20], v[19:20], 1
-; SDAG-NEXT: v_or_b32_e32 v21, v21, v23
-; SDAG-NEXT: v_lshrrev_b32_e32 v23, 31, v1
-; SDAG-NEXT: v_or_b32_e32 v19, v19, v23
-; SDAG-NEXT: v_sub_i32_e32 v23, vcc, v18, v19
-; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v33, v20, vcc
-; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v34, v21, vcc
-; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v35, v22, vcc
-; SDAG-NEXT: v_ashrrev_i32_e32 v36, 31, v23
-; SDAG-NEXT: v_and_b32_e32 v23, v36, v27
+; SDAG-NEXT: v_lshl_b64 v[34:35], v[8:9], 1
+; SDAG-NEXT: v_lshl_b64 v[18:19], v[18:19], 1
+; SDAG-NEXT: v_or_b32_e32 v8, v20, v34
+; SDAG-NEXT: v_lshrrev_b32_e32 v20, 31, v17
+; SDAG-NEXT: v_lshl_b64 v[16:17], v[16:17], 1
+; SDAG-NEXT: v_or_b32_e32 v18, v18, v20
+; SDAG-NEXT: v_lshrrev_b32_e32 v20, 31, v1
+; SDAG-NEXT: v_or_b32_e32 v16, v16, v20
+; SDAG-NEXT: v_sub_i32_e32 v20, vcc, v30, v16
+; SDAG-NEXT: v_subb_u32_e32 v20, vcc, v31, v17, vcc
+; SDAG-NEXT: v_subb_u32_e32 v20, vcc, v32, v18, vcc
+; SDAG-NEXT: v_subb_u32_e32 v20, vcc, v33, v19, vcc
+; SDAG-NEXT: v_ashrrev_i32_e32 v34, 31, v20
+; SDAG-NEXT: v_and_b32_e32 v20, v34, v24
; SDAG-NEXT: v_lshrrev_b32_e32 v10, 31, v9
-; SDAG-NEXT: v_or_b32_e32 v9, v24, v37
-; SDAG-NEXT: v_and_b32_e32 v24, v36, v28
-; SDAG-NEXT: v_sub_i32_e32 v19, vcc, v19, v23
-; SDAG-NEXT: v_subb_u32_e32 v20, vcc, v20, v24, vcc
-; SDAG-NEXT: v_and_b32_e32 v23, v36, v2
-; SDAG-NEXT: v_and_b32_e32 v24, v36, v3
-; SDAG-NEXT: v_subb_u32_e32 v21, vcc, v21, v23, vcc
-; SDAG-NEXT: v_subb_u32_e32 v22, vcc, v22, v24, vcc
-; SDAG-NEXT: v_add_i32_e32 v29, vcc, -1, v29
-; SDAG-NEXT: v_addc_u32_e32 v30, vcc, -1, v30, vcc
-; SDAG-NEXT: v_addc_u32_e32 v31, vcc, -1, v31, vcc
-; SDAG-NEXT: v_addc_u32_e32 v32, vcc, -1, v32, vcc
-; SDAG-NEXT: v_or_b32_e32 v23, v29, v31
-; SDAG-NEXT: v_or_b32_e32 v24, v30, v32
+; SDAG-NEXT: v_or_b32_e32 v9, v21, v35
+; SDAG-NEXT: v_and_b32_e32 v21, v34, v25
+; SDAG-NEXT: v_sub_i32_e32 v16, vcc, v16, v20
+; SDAG-NEXT: v_subb_u32_e32 v17, vcc, v17, v21, vcc
+; SDAG-NEXT: v_and_b32_e32 v20, v34, v2
+; SDAG-NEXT: v_and_b32_e32 v21, v34, v3
+; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v18, v20, vcc
+; SDAG-NEXT: v_subb_u32_e32 v19, vcc, v19, v21, vcc
+; SDAG-NEXT: v_add_i32_e32 v26, vcc, -1, v26
+; SDAG-NEXT: v_addc_u32_e32 v27, vcc, -1, v27, vcc
+; SDAG-NEXT: v_addc_u32_e32 v28, vcc, -1, v28, vcc
+; SDAG-NEXT: v_addc_u32_e32 v29, vcc, -1, v29, vcc
+; SDAG-NEXT: v_or_b32_e32 v20, v26, v28
+; SDAG-NEXT: v_or_b32_e32 v21, v27, v29
; SDAG-NEXT: v_lshl_b64 v[0:1], v[0:1], 1
-; SDAG-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[23:24]
+; SDAG-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[20:21]
+; SDAG-NEXT: v_mov_b32_e32 v20, 0
+; SDAG-NEXT: v_mov_b32_e32 v21, 0
; SDAG-NEXT: v_or_b32_e32 v0, v0, v10
-; SDAG-NEXT: v_and_b32_e32 v10, 1, v36
-; SDAG-NEXT: v_or_b32_e32 v1, v17, v1
-; SDAG-NEXT: v_or_b32_e32 v0, v16, v0
+; SDAG-NEXT: v_and_b32_e32 v10, 1, v34
+; SDAG-NEXT: v_or_b32_e32 v1, v21, v1
+; SDAG-NEXT: v_or_b32_e32 v0, v20, v0
; SDAG-NEXT: s_or_b64 s[4:5], vcc, s[4:5]
-; SDAG-NEXT: v_mov_b32_e32 v24, v11
-; SDAG-NEXT: v_mov_b32_e32 v23, v10
+; SDAG-NEXT: v_mov_b32_e32 v21, v11
+; SDAG-NEXT: v_mov_b32_e32 v20, v10
; SDAG-NEXT: s_andn2_b64 exec, exec, s[4:5]
; SDAG-NEXT: s_cbranch_execnz .LBB0_3
; SDAG-NEXT: ; %bb.4: ; %Flow13
@@ -205,14 +205,14 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_subb_u32_e32 v7, vcc, v2, v18, vcc
; SDAG-NEXT: v_xor_b32_e32 v3, v12, v19
; SDAG-NEXT: v_xor_b32_e32 v2, v13, v19
-; SDAG-NEXT: v_sub_i32_e32 v22, vcc, v3, v19
-; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v2, v19, vcc
+; SDAG-NEXT: v_sub_i32_e32 v24, vcc, v3, v19
+; SDAG-NEXT: v_subb_u32_e32 v25, vcc, v2, v19, vcc
; SDAG-NEXT: v_xor_b32_e32 v2, v14, v19
; SDAG-NEXT: v_xor_b32_e32 v3, v15, v19
; SDAG-NEXT: v_subb_u32_e32 v2, vcc, v2, v19, vcc
; SDAG-NEXT: v_subb_u32_e32 v3, vcc, v3, v19, vcc
-; SDAG-NEXT: v_or_b32_e32 v9, v23, v3
-; SDAG-NEXT: v_or_b32_e32 v8, v22, v2
+; SDAG-NEXT: v_or_b32_e32 v9, v25, v3
+; SDAG-NEXT: v_or_b32_e32 v8, v24, v2
; SDAG-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[8:9]
; SDAG-NEXT: v_or_b32_e32 v9, v5, v7
; SDAG-NEXT: v_or_b32_e32 v8, v4, v6
@@ -222,9 +222,9 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_add_i32_e32 v8, vcc, 32, v8
; SDAG-NEXT: v_ffbh_u32_e32 v9, v3
; SDAG-NEXT: v_min_u32_e32 v8, v8, v9
-; SDAG-NEXT: v_ffbh_u32_e32 v9, v22
+; SDAG-NEXT: v_ffbh_u32_e32 v9, v24
; SDAG-NEXT: v_add_i32_e32 v9, vcc, 32, v9
-; SDAG-NEXT: v_ffbh_u32_e32 v10, v23
+; SDAG-NEXT: v_ffbh_u32_e32 v10, v25
; SDAG-NEXT: v_min_u32_e32 v9, v9, v10
; SDAG-NEXT: v_add_i32_e32 v9, vcc, 64, v9
; SDAG-NEXT: v_addc_u32_e64 v10, s[6:7], 0, 0, vcc
@@ -272,7 +272,7 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: s_and_saveexec_b64 s[6:7], s[4:5]
; SDAG-NEXT: s_cbranch_execz .LBB0_12
; SDAG-NEXT: ; %bb.7: ; %udiv-bb1
-; SDAG-NEXT: v_add_i32_e32 v24, vcc, 1, v8
+; SDAG-NEXT: v_add_i32_e32 v26, vcc, 1, v8
; SDAG-NEXT: v_addc_u32_e32 v27, vcc, 0, v9, vcc
; SDAG-NEXT: v_addc_u32_e32 v28, vcc, 0, v10, vcc
; SDAG-NEXT: v_addc_u32_e64 v29, s[4:5], 0, v11, vcc
@@ -300,23 +300,23 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: s_xor_b64 s[8:9], exec, s[4:5]
; SDAG-NEXT: s_cbranch_execz .LBB0_11
; SDAG-NEXT: ; %bb.8: ; %udiv-preheader
-; SDAG-NEXT: v_sub_i32_e32 v14, vcc, 64, v24
-; SDAG-NEXT: v_lshr_b64 v[12:13], v[4:5], v24
+; SDAG-NEXT: v_sub_i32_e32 v14, vcc, 64, v26
+; SDAG-NEXT: v_lshr_b64 v[12:13], v[4:5], v26
; SDAG-NEXT: v_lshl_b64 v[14:15], v[6:7], v14
-; SDAG-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v24
+; SDAG-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v26
; SDAG-NEXT: v_or_b32_e32 v14, v12, v14
-; SDAG-NEXT: v_subrev_i32_e32 v12, vcc, 64, v24
+; SDAG-NEXT: v_subrev_i32_e32 v12, vcc, 64, v26
; SDAG-NEXT: v_or_b32_e32 v15, v13, v15
; SDAG-NEXT: v_lshr_b64 v[12:13], v[6:7], v12
-; SDAG-NEXT: v_cmp_gt_u32_e32 vcc, 64, v24
+; SDAG-NEXT: v_cmp_gt_u32_e32 vcc, 64, v26
; SDAG-NEXT: v_cndmask_b32_e32 v13, v13, v15, vcc
; SDAG-NEXT: v_cndmask_b32_e64 v15, v13, v5, s[4:5]
-; SDAG-NEXT: v_lshr_b64 v[5:6], v[6:7], v24
+; SDAG-NEXT: v_lshr_b64 v[5:6], v[6:7], v26
; SDAG-NEXT: v_cndmask_b32_e32 v12, v12, v14, vcc
; SDAG-NEXT: v_cndmask_b32_e32 v7, 0, v6, vcc
; SDAG-NEXT: v_cndmask_b32_e32 v6, 0, v5, vcc
-; SDAG-NEXT: v_add_i32_e32 v30, vcc, -1, v22
-; SDAG-NEXT: v_addc_u32_e32 v31, vcc, -1, v23, vcc
+; SDAG-NEXT: v_add_i32_e32 v30, vcc, -1, v24
+; SDAG-NEXT: v_addc_u32_e32 v31, vcc, -1, v25, vcc
; SDAG-NEXT: v_addc_u32_e32 v32, vcc, -1, v2, vcc
; SDAG-NEXT: v_cndmask_b32_e64 v14, v12, v4, s[4:5]
; SDAG-NEXT: v_addc_u32_e32 v33, vcc, -1, v3, vcc
@@ -342,7 +342,7 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_lshl_b64 v[10:11], v[10:11], 1
; SDAG-NEXT: v_subb_u32_e32 v12, vcc, v32, v6, vcc
; SDAG-NEXT: v_subb_u32_e32 v12, vcc, v33, v7, vcc
-; SDAG-NEXT: v_add_i32_e64 v24, s[4:5], -1, v24
+; SDAG-NEXT: v_add_i32_e64 v26, s[4:5], -1, v26
; SDAG-NEXT: v_or_b32_e32 v10, v16, v10
; SDAG-NEXT: v_ashrrev_i32_e32 v16, 31, v12
; SDAG-NEXT: v_addc_u32_e64 v27, s[4:5], -1, v27, s[4:5]
@@ -350,14 +350,14 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_and_b32_e32 v12, 1, v16
; SDAG-NEXT: v_and_b32_e32 v34, v16, v3
; SDAG-NEXT: v_and_b32_e32 v35, v16, v2
-; SDAG-NEXT: v_and_b32_e32 v17, v16, v23
-; SDAG-NEXT: v_and_b32_e32 v16, v16, v22
+; SDAG-NEXT: v_and_b32_e32 v17, v16, v25
+; SDAG-NEXT: v_and_b32_e32 v16, v16, v24
; SDAG-NEXT: v_addc_u32_e64 v28, s[4:5], -1, v28, s[4:5]
; SDAG-NEXT: v_sub_i32_e32 v14, vcc, v14, v16
; SDAG-NEXT: v_addc_u32_e64 v29, s[4:5], -1, v29, s[4:5]
; SDAG-NEXT: v_subb_u32_e32 v15, vcc, v15, v17, vcc
; SDAG-NEXT: v_or_b32_e32 v17, v27, v29
-; SDAG-NEXT: v_or_b32_e32 v16, v24, v28
+; SDAG-NEXT: v_or_b32_e32 v16, v26, v28
; SDAG-NEXT: v_cmp_eq_u64_e64 s[4:5], 0, v[16:17]
; SDAG-NEXT: v_subb_u32_e32 v6, vcc, v6, v35, vcc
; SDAG-NEXT: v_or_b32_e32 v9, v5, v9
@@ -380,7 +380,7 @@ define <2 x i128> @v_sdiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_or_b32_e32 v12, v12, v2
; SDAG-NEXT: .LBB0_12: ; %Flow12
; SDAG-NEXT: s_or_b64 exec, exec, s[6:7]
-; SDAG-NEXT: v_xor_b32_e32 v3, v26, v25
+; SDAG-NEXT: v_xor_b32_e32 v3, v23, v22
; SDAG-NEXT: v_xor_b32_e32 v2, v0, v3
; SDAG-NEXT: v_xor_b32_e32 v0, v21, v3
; SDAG-NEXT: v_xor_b32_e32 v4, v1, v3
@@ -987,10 +987,9 @@ define <2 x i128> @v_udiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_mov_b32_e32 v21, 0
; SDAG-NEXT: .LBB2_3: ; %udiv-do-while3
; SDAG-NEXT: ; =>This Inner Loop Header: Depth=1
-; SDAG-NEXT: v_lshrrev_b32_e32 v20, 31, v17
-; SDAG-NEXT: v_lshl_b64 v[16:17], v[16:17], 1
+; SDAG-NEXT: v_lshl_b64 v[34:35], v[16:17], 1
; SDAG-NEXT: v_lshl_b64 v[18:19], v[18:19], 1
-; SDAG-NEXT: v_or_b32_e32 v16, v24, v16
+; SDAG-NEXT: v_or_b32_e32 v16, v24, v34
; SDAG-NEXT: v_lshrrev_b32_e32 v24, 31, v23
; SDAG-NEXT: v_lshl_b64 v[22:23], v[22:23], 1
; SDAG-NEXT: v_or_b32_e32 v18, v18, v24
@@ -1002,18 +1001,19 @@ define <2 x i128> @v_udiv_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_subb_u32_e32 v24, vcc, v33, v19, vcc
; SDAG-NEXT: v_ashrrev_i32_e32 v34, 31, v24
; SDAG-NEXT: v_and_b32_e32 v24, v34, v8
+; SDAG-NEXT: v_lshrrev_b32_e32 v20, 31, v17
+; SDAG-NEXT: v_or_b32_e32 v17, v25, v35
+; SDAG-NEXT: v_and_b32_e32 v25, v34, v9
; SDAG-NEXT: v_sub_i32_e32 v22, vcc, v22, v24
-; SDAG-NEXT: v_and_b32_e32 v24, v34, v9
-; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v23, v24, vcc
+; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v23, v25, vcc
; SDAG-NEXT: v_and_b32_e32 v24, v34, v10
+; SDAG-NEXT: v_and_b32_e32 v25, v34, v11
; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v18, v24, vcc
-; SDAG-NEXT: v_and_b32_e32 v24, v34, v11
-; SDAG-NEXT: v_subb_u32_e32 v19, vcc, v19, v24, vcc
+; SDAG-NEXT: v_subb_u32_e32 v19, vcc, v19, v25, vcc
; SDAG-NEXT: v_add_i32_e32 v26, vcc, -1, v26
; SDAG-NEXT: v_addc_u32_e32 v27, vcc, -1, v27, vcc
; SDAG-NEXT: v_addc_u32_e32 v28, vcc, -1, v28, vcc
; SDAG-NEXT: v_addc_u32_e32 v29, vcc, -1, v29, vcc
-; SDAG-NEXT: v_or_b32_e32 v17, v25, v17
; SDAG-NEXT: v_or_b32_e32 v24, v26, v28
; SDAG-NEXT: v_or_b32_e32 v25, v27, v29
; SDAG-NEXT: v_lshl_b64 v[2:3], v[2:3], 1
@@ -1743,54 +1743,54 @@ define <2 x i128> @v_srem_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_addc_u32_e32 v38, vcc, -1, v9, vcc
; SDAG-NEXT: v_mov_b32_e32 v20, 0
; SDAG-NEXT: v_mov_b32_e32 v21, 0
-; SDAG-NEXT: s_mov_b64 s[10:11], 0
+; SDAG-NEXT: s_mov_b64 s[4:5], 0
; SDAG-NEXT: v_mov_b32_e32 v26, 0
; SDAG-NEXT: v_mov_b32_e32 v27, 0
; SDAG-NEXT: v_mov_b32_e32 v19, 0
; SDAG-NEXT: .LBB4_3: ; %udiv-do-while3
; SDAG-NEXT: ; =>This Inner Loop Header: Depth=1
-; SDAG-NEXT: v_lshrrev_b32_e32 v18, 31, v17
-; SDAG-NEXT: v_lshl_b64 v[16:17], v[16:17], 1
-; SDAG-NEXT: v_add_i32_e32 v31, vcc, -1, v31
-; SDAG-NEXT: v_or_b32_e32 v16, v26, v16
+; SDAG-NEXT: v_lshl_b64 v[48:49], v[16:17], 1
+; SDAG-NEXT: v_lshl_b64 v[24:25], v[24:25], 1
+; SDAG-NEXT: v_or_b32_e32 v16, v26, v48
; SDAG-NEXT: v_lshrrev_b32_e32 v26, 31, v23
; SDAG-NEXT: v_lshl_b64 v[22:23], v[22:23], 1
+; SDAG-NEXT: v_or_b32_e32 v24, v24, v26
+; SDAG-NEXT: v_lshrrev_b32_e32 v26, 31, v11
+; SDAG-NEXT: v_or_b32_e32 v22, v22, v26
+; SDAG-NEXT: v_sub_i32_e32 v26, vcc, v35, v22
+; SDAG-NEXT: v_subb_u32_e32 v26, vcc, v36, v23, vcc
+; SDAG-NEXT: v_subb_u32_e32 v26, vcc, v37, v24, vcc
+; SDAG-NEXT: v_subb_u32_e32 v26, vcc, v38, v25, vcc
+; SDAG-NEXT: v_ashrrev_i32_e32 v39, 31, v26
+; SDAG-NEXT: v_lshrrev_b32_e32 v18, 31, v17
+; SDAG-NEXT: v_or_b32_e32 v17, v27, v49
+; SDAG-NEXT: v_and_b32_e32 v27, v39, v30
+; SDAG-NEXT: v_and_b32_e32 v26, v39, v29
+; SDAG-NEXT: v_sub_i32_e32 v22, vcc, v22, v27
+; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v23, v26, vcc
+; SDAG-NEXT: v_and_b32_e32 v27, v39, v8
+; SDAG-NEXT: v_and_b32_e32 v26, v39, v9
+; SDAG-NEXT: v_subb_u32_e32 v24, vcc, v24, v27, vcc
+; SDAG-NEXT: v_subb_u32_e32 v25, vcc, v25, v26, vcc
+; SDAG-NEXT: v_add_i32_e32 v31, vcc, -1, v31
; SDAG-NEXT: v_addc_u32_e32 v32, vcc, -1, v32, vcc
-; SDAG-NEXT: v_or_b32_e32 v17, v27, v17
-; SDAG-NEXT: v_lshrrev_b32_e32 v27, 31, v11
-; SDAG-NEXT: v_lshl_b64 v[10:11], v[10:11], 1
; SDAG-NEXT: v_addc_u32_e32 v33, vcc, -1, v33, vcc
-; SDAG-NEXT: v_lshl_b64 v[24:25], v[24:25], 1
-; SDAG-NEXT: v_or_b32_e32 v22, v22, v27
; SDAG-NEXT: v_addc_u32_e32 v34, vcc, -1, v34, vcc
-; SDAG-NEXT: v_or_b32_e32 v10, v10, v18
-; SDAG-NEXT: v_sub_i32_e32 v18, vcc, v35, v22
-; SDAG-NEXT: v_or_b32_e32 v24, v24, v26
-; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v36, v23, vcc
-; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v37, v24, vcc
-; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v38, v25, vcc
-; SDAG-NEXT: v_or_b32_e32 v26, v31, v33
; SDAG-NEXT: v_or_b32_e32 v27, v32, v34
-; SDAG-NEXT: v_ashrrev_i32_e32 v18, 31, v18
-; SDAG-NEXT: v_cmp_eq_u64_e64 s[4:5], 0, v[26:27]
-; SDAG-NEXT: v_and_b32_e32 v27, v18, v30
-; SDAG-NEXT: v_and_b32_e32 v26, v18, v29
-; SDAG-NEXT: v_sub_i32_e32 v22, vcc, v22, v27
-; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v23, v26, vcc
-; SDAG-NEXT: v_and_b32_e32 v48, v18, v8
-; SDAG-NEXT: v_and_b32_e32 v39, v18, v9
-; SDAG-NEXT: v_and_b32_e32 v18, 1, v18
-; SDAG-NEXT: v_subb_u32_e32 v24, vcc, v24, v48, vcc
+; SDAG-NEXT: v_or_b32_e32 v26, v31, v33
+; SDAG-NEXT: v_lshl_b64 v[10:11], v[10:11], 1
+; SDAG-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[26:27]
+; SDAG-NEXT: v_or_b32_e32 v10, v10, v18
+; SDAG-NEXT: v_and_b32_e32 v18, 1, v39
; SDAG-NEXT: v_or_b32_e32 v11, v21, v11
; SDAG-NEXT: v_or_b32_e32 v10, v20, v10
-; SDAG-NEXT: s_or_b64 s[10:11], s[4:5], s[10:11]
+; SDAG-NEXT: s_or_b64 s[4:5], vcc, s[4:5]
; SDAG-NEXT: v_mov_b32_e32 v27, v19
; SDAG-NEXT: v_mov_b32_e32 v26, v18
-; SDAG-NEXT: v_subb_u32_e32 v25, vcc, v25, v39, vcc
-; SDAG-NEXT: s_andn2_b64 exec, exec, s[10:11]
+; SDAG-NEXT: s_andn2_b64 exec, exec, s[4:5]
; SDAG-NEXT: s_cbranch_execnz .LBB4_3
; SDAG-NEXT: ; %bb.4: ; %Flow13
-; SDAG-NEXT: s_or_b64 exec, exec, s[10:11]
+; SDAG-NEXT: s_or_b64 exec, exec, s[4:5]
; SDAG-NEXT: .LBB4_5: ; %Flow14
; SDAG-NEXT: s_or_b64 exec, exec, s[8:9]
; SDAG-NEXT: v_lshl_b64 v[10:11], v[10:11], 1
@@ -1837,73 +1837,73 @@ define <2 x i128> @v_srem_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_add_i32_e32 v15, vcc, 64, v15
; SDAG-NEXT: v_addc_u32_e64 v16, s[6:7], 0, 0, vcc
; SDAG-NEXT: v_cmp_ne_u64_e32 vcc, 0, v[12:13]
-; SDAG-NEXT: v_ffbh_u32_e32 v18, v5
+; SDAG-NEXT: v_ffbh_u32_e32 v17, v7
; SDAG-NEXT: v_cndmask_b32_e32 v14, v15, v14, vcc
; SDAG-NEXT: v_ffbh_u32_e32 v15, v6
-; SDAG-NEXT: v_cndmask_b32_e64 v17, v16, 0, vcc
+; SDAG-NEXT: v_cndmask_b32_e64 v16, v16, 0, vcc
; SDAG-NEXT: v_add_i32_e32 v15, vcc, 32, v15
-; SDAG-NEXT: v_ffbh_u32_e32 v16, v7
-; SDAG-NEXT: v_min_u32_e32 v15, v15, v16
-; SDAG-NEXT: v_ffbh_u32_e32 v16, v4
-; SDAG-NEXT: v_add_i32_e32 v16, vcc, 32, v16
-; SDAG-NEXT: v_min_u32_e32 v16, v16, v18
-; SDAG-NEXT: v_add_i32_e32 v16, vcc, 64, v16
+; SDAG-NEXT: v_min_u32_e32 v15, v15, v17
+; SDAG-NEXT: v_ffbh_u32_e32 v17, v4
+; SDAG-NEXT: v_add_i32_e32 v17, vcc, 32, v17
+; SDAG-NEXT: v_ffbh_u32_e32 v18, v5
+; SDAG-NEXT: v_min_u32_e32 v17, v17, v18
+; SDAG-NEXT: v_add_i32_e32 v17, vcc, 64, v17
; SDAG-NEXT: v_addc_u32_e64 v18, s[6:7], 0, 0, vcc
; SDAG-NEXT: v_cmp_ne_u64_e32 vcc, 0, v[6:7]
; SDAG-NEXT: s_movk_i32 s8, 0x7f
-; SDAG-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
+; SDAG-NEXT: v_cndmask_b32_e32 v15, v17, v15, vcc
; SDAG-NEXT: v_cndmask_b32_e64 v18, v18, 0, vcc
-; SDAG-NEXT: v_sub_i32_e32 v16, vcc, v14, v15
-; SDAG-NEXT: v_subb_u32_e32 v17, vcc, v17, v18, vcc
+; SDAG-NEXT: v_sub_i32_e32 v14, vcc, v14, v15
+; SDAG-NEXT: v_subb_u32_e32 v15, vcc, v16, v18, vcc
; SDAG-NEXT: v_subb_u32_e64 v18, s[6:7], 0, 0, vcc
; SDAG-NEXT: v_subb_u32_e64 v19, s[6:7], 0, 0, s[6:7]
; SDAG-NEXT: s_mov_b64 s[6:7], 0x7f
-; SDAG-NEXT: v_cmp_lt_u64_e32 vcc, s[6:7], v[16:17]
-; SDAG-NEXT: v_or_b32_e32 v21, v17, v19
-; SDAG-NEXT: v_cndmask_b32_e64 v14, 0, 1, vcc
+; SDAG-NEXT: v_cmp_lt_u64_e32 vcc, s[6:7], v[14:15]
+; SDAG-NEXT: v_or_b32_e32 v21, v15, v19
+; SDAG-NEXT: v_cndmask_b32_e64 v16, 0, 1, vcc
; SDAG-NEXT: v_cmp_ne_u64_e32 vcc, 0, v[18:19]
-; SDAG-NEXT: v_cndmask_b32_e64 v15, 0, 1, vcc
+; SDAG-NEXT: v_cndmask_b32_e64 v17, 0, 1, vcc
; SDAG-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[18:19]
-; SDAG-NEXT: v_cndmask_b32_e32 v14, v15, v14, vcc
-; SDAG-NEXT: v_and_b32_e32 v14, 1, v14
-; SDAG-NEXT: v_cmp_eq_u32_e32 vcc, 1, v14
-; SDAG-NEXT: v_xor_b32_e32 v14, 0x7f, v16
-; SDAG-NEXT: v_or_b32_e32 v20, v14, v18
+; SDAG-NEXT: v_cndmask_b32_e32 v16, v17, v16, vcc
+; SDAG-NEXT: v_and_b32_e32 v16, 1, v16
+; SDAG-NEXT: v_cmp_eq_u32_e32 vcc, 1, v16
+; SDAG-NEXT: v_xor_b32_e32 v16, 0x7f, v14
+; SDAG-NEXT: v_or_b32_e32 v20, v16, v18
; SDAG-NEXT: s_or_b64 s[4:5], s[4:5], vcc
; SDAG-NEXT: v_cmp_ne_u64_e32 vcc, 0, v[20:21]
; SDAG-NEXT: s_xor_b64 s[6:7], s[4:5], -1
-; SDAG-NEXT: v_cndmask_b32_e64 v15, v7, 0, s[4:5]
-; SDAG-NEXT: v_cndmask_b32_e64 v14, v6, 0, s[4:5]
+; SDAG-NEXT: v_cndmask_b32_e64 v17, v7, 0, s[4:5]
+; SDAG-NEXT: v_cndmask_b32_e64 v16, v6, 0, s[4:5]
; SDAG-NEXT: v_cndmask_b32_e64 v20, v5, 0, s[4:5]
; SDAG-NEXT: v_cndmask_b32_e64 v21, v4, 0, s[4:5]
; SDAG-NEXT: s_and_b64 s[4:5], s[6:7], vcc
; SDAG-NEXT: s_and_saveexec_b64 s[6:7], s[4:5]
; SDAG-NEXT: s_cbranch_execz .LBB4_12
; SDAG-NEXT: ; %bb.7: ; %udiv-bb1
-; SDAG-NEXT: v_add_i32_e32 v36, vcc, 1, v16
-; SDAG-NEXT: v_addc_u32_e32 v37, vcc, 0, v17, vcc
+; SDAG-NEXT: v_add_i32_e32 v36, vcc, 1, v14
+; SDAG-NEXT: v_addc_u32_e32 v37, vcc, 0, v15, vcc
; SDAG-NEXT: v_addc_u32_e32 v38, vcc, 0, v18, vcc
; SDAG-NEXT: v_addc_u32_e64 v39, s[4:5], 0, v19, vcc
-; SDAG-NEXT: v_sub_i32_e32 v19, vcc, s8, v16
+; SDAG-NEXT: v_sub_i32_e32 v19, vcc, s8, v14
; SDAG-NEXT: v_sub_i32_e32 v17, vcc, 64, v19
-; SDAG-NEXT: v_lshl_b64 v[14:15], v[6:7], v19
+; SDAG-NEXT: v_lshl_b64 v[15:16], v[6:7], v19
; SDAG-NEXT: v_lshr_b64 v[17:18], v[4:5], v17
-; SDAG-NEXT: s_xor_b64 s[8:9], s[4:5], -1
-; SDAG-NEXT: v_or_b32_e32 v17, v14, v17
-; SDAG-NEXT: v_sub_i32_e32 v14, vcc, 63, v16
-; SDAG-NEXT: v_or_b32_e32 v18, v15, v18
+; SDAG-NEXT: v_sub_i32_e32 v14, vcc, 63, v14
+; SDAG-NEXT: v_or_b32_e32 v16, v16, v18
+; SDAG-NEXT: v_or_b32_e32 v18, v15, v17
; SDAG-NEXT: v_lshl_b64 v[14:15], v[4:5], v14
; SDAG-NEXT: v_cmp_gt_u32_e32 vcc, 64, v19
-; SDAG-NEXT: v_cndmask_b32_e32 v14, v14, v17, vcc
-; SDAG-NEXT: v_lshl_b64 v[16:17], v[4:5], v19
-; SDAG-NEXT: v_cndmask_b32_e32 v15, v15, v18, vcc
+; SDAG-NEXT: s_xor_b64 s[8:9], s[4:5], -1
+; SDAG-NEXT: v_cndmask_b32_e32 v15, v15, v16, vcc
; SDAG-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v19
-; SDAG-NEXT: v_cndmask_b32_e64 v15, v15, v7, s[4:5]
-; SDAG-NEXT: v_cndmask_b32_e64 v14, v14, v6, s[4:5]
-; SDAG-NEXT: v_cndmask_b32_e32 v17, 0, v17, vcc
+; SDAG-NEXT: v_cndmask_b32_e64 v17, v15, v7, s[4:5]
+; SDAG-NEXT: v_cndmask_b32_e32 v16, v14, v18, vcc
+; SDAG-NEXT: v_lshl_b64 v[14:15], v[4:5], v19
+; SDAG-NEXT: v_cndmask_b32_e64 v16, v16, v6, s[4:5]
+; SDAG-NEXT: v_cndmask_b32_e32 v15, 0, v15, vcc
; SDAG-NEXT: v_mov_b32_e32 v18, 0
; SDAG-NEXT: v_mov_b32_e32 v19, 0
-; SDAG-NEXT: v_cndmask_b32_e32 v16, 0, v16, vcc
+; SDAG-NEXT: v_cndmask_b32_e32 v14, 0, v14, vcc
; SDAG-NEXT: s_and_saveexec_b64 s[4:5], s[8:9]
; SDAG-NEXT: s_xor_b64 s[8:9], exec, s[4:5]
; SDAG-NEXT: s_cbranch_execz .LBB4_11
@@ -1936,87 +1936,87 @@ define <2 x i128> @v_srem_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_mov_b32_e32 v19, 0
; SDAG-NEXT: .LBB4_9: ; %udiv-do-while
; SDAG-NEXT: ; =>This Inner Loop Header: Depth=1
-; SDAG-NEXT: v_lshl_b64 v[26:27], v[26:27], 1
-; SDAG-NEXT: v_lshrrev_b32_e32 v18, 31, v23
+; SDAG-NEXT: v_lshl_b64 v[52:53], v[14:15], 1
+; SDAG-NEXT: v_add_i32_e32 v36, vcc, -1, v36
+; SDAG-NEXT: v_or_b32_e32 v14, v24, v52
+; SDAG-NEXT: v_lshrrev_b32_e32 v24, 31, v23
; SDAG-NEXT: v_lshl_b64 v[22:23], v[22:23], 1
-; SDAG-NEXT: v_or_b32_e32 v26, v26, v18
+; SDAG-NEXT: v_addc_u32_e32 v37, vcc, -1, v37, vcc
; SDAG-NEXT: v_lshrrev_b32_e32 v18, 31, v15
-; SDAG-NEXT: v_lshl_b64 v[14:15], v[14:15], 1
-; SDAG-NEXT: v_or_b32_e32 v22, v22, v18
-; SDAG-NEXT: v_lshrrev_b32_e32 v18, 31, v17
-; SDAG-NEXT: v_or_b32_e32 v14, v14, v18
+; SDAG-NEXT: v_or_b32_e32 v15, v25, v53
+; SDAG-NEXT: v_lshrrev_b32_e32 v25, 31, v17
+; SDAG-NEXT: v_lshl_b64 v[16:17], v[16:17], 1
+; SDAG-NEXT: v_addc_u32_e32 v38, vcc, -1, v38, vcc
+; SDAG-NEXT: v_lshl_b64 v[26:27], v[26:27], 1
+; SDAG-NEXT: v_or_b32_e32 v22, v22, v25
+; SDAG-NEXT: v_addc_u32_e32 v39, vcc, -1, v39, vcc
+; SDAG-NEXT: v_or_b32_e32 v16, v16, v18
; SDAG-NEXT: v_sub_i32_e32 v18, vcc, v48, v22
+; SDAG-NEXT: v_or_b32_e32 v26, v26, v24
; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v49, v23, vcc
-; SDAG-NEXT: v_lshl_b64 v[16:17], v[16:17], 1
; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v50, v26, vcc
+; SDAG-NEXT: v_or_b32_e32 v24, v36, v38
+; SDAG-NEXT: v_or_b32_e32 v25, v37, v39
; SDAG-NEXT: v_subb_u32_e32 v18, vcc, v51, v27, vcc
-; SDAG-NEXT: v_add_i32_e64 v36, s[4:5], -1, v36
-; SDAG-NEXT: v_or_b32_e32 v16, v24, v16
+; SDAG-NEXT: v_cmp_eq_u64_e64 s[4:5], 0, v[24:25]
; SDAG-NEXT: v_ashrrev_i32_e32 v24, 31, v18
-; SDAG-NEXT: v_addc_u32_e64 v37, s[4:5], -1, v37, s[4:5]
-; SDAG-NEXT: v_or_b32_e32 v17, v25, v17
-; SDAG-NEXT: v_and_b32_e32 v18, 1, v24
-; SDAG-NEXT: v_and_b32_e32 v52, v24, v13
-; SDAG-NEXT: v_and_b32_e32 v53, v24, v12
+; SDAG-NEXT: v_and_b32_e32 v52, v24, v33
; SDAG-NEXT: v_and_b32_e32 v25, v24, v32
-; SDAG-NEXT: v_and_b32_e32 v24, v24, v33
-; SDAG-NEXT: v_addc_u32_e64 v38, s[4:5], -1, v38, s[4:5]
-; SDAG-NEXT: v_sub_i32_e32 v22, vcc, v22, v24
-; SDAG-NEXT: v_addc_u32_e64 v39, s[4:5], -1, v39, s[4:5]
+; SDAG-NEXT: v_sub_i32_e32 v22, vcc, v22, v52
; SDAG-NEXT: v_subb_u32_e32 v23, vcc, v23, v25, vcc
-; SDAG-NEXT: v_or_b32_e32 v25, v37, v39
-; SDAG-NEXT: v_or_b32_e32 v24, v36, v38
-; SDAG-NEXT: v_cmp_eq_u64_e64 s[4:5], 0, v[24:25]
+; SDAG-NEXT: v_and_b32_e32 v53, v24, v12
+; SDAG-NEXT: v_and_b32_e32 v18, 1, v24
+; SDAG-NEXT: v_and_b32_e32 v52, v24, v13
; SDAG-NEXT: v_subb_u32_e32 v26, vcc, v26, v53, vcc
-; SDAG-NEXT: v_or_b32_e32 v15, v21, v15
-; SDAG-NEXT: v_or_b32_e32 v14, v20, v14
-; SDAG-NEXT: v_subb_u32_e32 v27, vcc, v27, v52, vcc
+; SDAG-NEXT: v_or_b32_e32 v17, v21, v17
+; SDAG-NEXT: v_or_b32_e32 v16, v20, v16
; SDAG-NEXT: s_or_b64 s[10:11], s[4:5], s[10:11]
; SDAG-NEXT: v_mov_b32_e32 v25, v19
; SDAG-NEXT: v_mov_b32_e32 v24, v18
+; SDAG-NEXT: v_subb_u32_e32 v27, vcc, v27, v52, vcc
; SDAG-NEXT: s_andn2_b64 exec, exec, s[10:11]
; SDAG-NEXT: s_cbranch_execnz .LBB4_9
; SDAG-NEXT: ; %bb.10: ; %Flow
; SDAG-NEXT: s_or_b64 exec, exec, s[10:11]
; SDAG-NEXT: .LBB4_11: ; %Flow11
; SDAG-NEXT: s_or_b64 exec, exec, s[8:9]
-; SDAG-NEXT: v_lshl_b64 v[14:15], v[14:15], 1
-; SDAG-NEXT: v_lshrrev_b32_e32 v20, 31, v17
; SDAG-NEXT: v_lshl_b64 v[16:17], v[16:17], 1
-; SDAG-NEXT: v_or_b32_e32 v14, v14, v20
-; SDAG-NEXT: v_or_b32_e32 v20, v19, v17
-; SDAG-NEXT: v_or_b32_e32 v21, v18, v16
+; SDAG-NEXT: v_lshrrev_b32_e32 v20, 31, v15
+; SDAG-NEXT: v_lshl_b64 v[14:15], v[14:15], 1
+; SDAG-NEXT: v_or_b32_e32 v16, v16, v20
+; SDAG-NEXT: v_or_b32_e32 v20, v19, v15
+; SDAG-NEXT: v_or_b32_e32 v21, v18, v14
; SDAG-NEXT: .LBB4_12: ; %Flow12
; SDAG-NEXT: s_or_b64 exec, exec, s[6:7]
; SDAG-NEXT: v_mul_lo_u32 v9, v35, v9
-; SDAG-NEXT: v_mad_u64_u32 v[22:23], s[4:5], v35, v8, 0
-; SDAG-NEXT: v_mad_u64_u32 v[16:17], s[4:5], v30, v35, 0
-; SDAG-NEXT: v_mov_b32_e32 v18, 0
+; SDAG-NEXT: v_mad_u64_u32 v[14:15], s[4:5], v35, v8, 0
+; SDAG-NEXT: v_mad_u64_u32 v[22:23], s[4:5], v30, v35, 0
+; SDAG-NEXT: v_mul_lo_u32 v18, v34, v8
+; SDAG-NEXT: v_mov_b32_e32 v24, 0
+; SDAG-NEXT: v_add_i32_e32 v15, vcc, v15, v9
+; SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v29, v35, v[23:24]
+; SDAG-NEXT: v_add_i32_e32 v15, vcc, v15, v18
+; SDAG-NEXT: v_mad_u64_u32 v[14:15], s[4:5], v10, v30, v[14:15]
; SDAG-NEXT: v_mul_lo_u32 v11, v11, v30
-; SDAG-NEXT: v_add_i32_e32 v19, vcc, v23, v9
-; SDAG-NEXT: v_mul_lo_u32 v23, v34, v8
-; SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v29, v35, v[17:18]
-; SDAG-NEXT: v_mul_lo_u32 v15, v15, v33
-; SDAG-NEXT: v_add_i32_e32 v23, vcc, v19, v23
-; SDAG-NEXT: v_mad_u64_u32 v[22:23], s[4:5], v10, v30, v[22:23]
-; SDAG-NEXT: v_mov_b32_e32 v17, v8
-; SDAG-NEXT: v_mad_u64_u32 v[24:25], s[4:5], v30, v34, v[17:18]
+; SDAG-NEXT: v_mov_b32_e32 v23, v8
+; SDAG-NEXT: v_mad_u64_u32 v[18:19], s[4:5], v30, v34, v[23:24]
; SDAG-NEXT: v_mul_lo_u32 v10, v10, v29
-; SDAG-NEXT: v_add_i32_e32 v11, vcc, v11, v23
-; SDAG-NEXT: v_add_i32_e32 v8, vcc, v9, v25
+; SDAG-NEXT: v_add_i32_e32 v11, vcc, v11, v15
+; SDAG-NEXT: v_add_i32_e32 v8, vcc, v9, v19
; SDAG-NEXT: v_addc_u32_e64 v9, s[4:5], 0, 0, vcc
; SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v29, v34, v[8:9]
; SDAG-NEXT: v_add_i32_e32 v10, vcc, v10, v11
-; SDAG-NEXT: v_add_i32_e32 v8, vcc, v8, v22
+; SDAG-NEXT: v_mul_lo_u32 v15, v16, v32
+; SDAG-NEXT: v_add_i32_e32 v8, vcc, v8, v14
; SDAG-NEXT: v_addc_u32_e32 v9, vcc, v9, v10, vcc
-; SDAG-NEXT: v_sub_i32_e32 v0, vcc, v0, v16
-; SDAG-NEXT: v_subb_u32_e32 v1, vcc, v1, v24, vcc
+; SDAG-NEXT: v_sub_i32_e32 v0, vcc, v0, v22
+; SDAG-NEXT: v_subb_u32_e32 v1, vcc, v1, v18, vcc
; SDAG-NEXT: v_subb_u32_e32 v2, vcc, v2, v8, vcc
; SDAG-NEXT: v_subb_u32_e32 v3, vcc, v3, v9, vcc
; SDAG-NEXT: v_xor_b32_e32 v0, v0, v28
; SDAG-NEXT: v_mul_lo_u32 v10, v21, v13
; SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v21, v12, 0
-; SDAG-NEXT: v_mad_u64_u32 v[16:17], s[4:5], v33, v21, 0
+; SDAG-NEXT: v_mad_u64_u32 v[22:23], s[4:5], v33, v21, 0
; SDAG-NEXT: v_xor_b32_e32 v1, v1, v28
; SDAG-NEXT: v_sub_i32_e32 v0, vcc, v0, v28
; SDAG-NEXT: v_xor_b32_e32 v2, v2, v28
@@ -2026,20 +2026,20 @@ define <2 x i128> @v_srem_v2i128_vv(<2 x i128> %lhs, <2 x i128> %rhs) {
; SDAG-NEXT: v_mul_lo_u32 v12, v20, v12
; SDAG-NEXT: v_subb_u32_e32 v3, vcc, v3, v28, vcc
; SDAG-NEXT: v_add_i32_e32 v9, vcc, v9, v10
-; SDAG-NEXT: v_mad_u64_u32 v[10:11], s[4:5], v32, v21, v[17:18]
+; SDAG-NEXT: v_mad_u64_u32 v[10:11], s[4:5], v32, v21, v[23:24]
; SDAG-NEXT: v_add_i32_e32 v9, vcc, v9, v12
-; SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v14, v33, v[8:9]
-; SDAG-NEXT: v_mov_b32_e32 v17, v10
-; SDAG-NEXT: v_mad_u64_u32 v[12:13], s[4:5], v33, v20, v[17:18]
-; SDAG-NEXT: v_mul_lo_u32 v14, v14, v32
-; SDAG-NEXT: v_add_i32_e32 v15, vcc, v15, v9
+; SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v16, v33, v[8:9]
+; SDAG-NEXT: v_mul_lo_u32 v14, v17, v33
+; SDAG-NEXT: v_mov_b32_e32 v23, v10
+; SDAG-NEXT: v_mad_u64_u32 v[12:13], s[4:5], v33, v20, v[23:24]
+; SDAG-NEXT: v_add_i32_e32 v14, vcc, v14, v9
; SDAG-NEXT: v_add_i32_e32 v9, vcc, v11, v13
; SDAG-NEXT: v_addc_u32_e64 v10, s[4:5], 0, 0, vcc
; SDAG-NEXT: v_mad_u64_u32 v[9:10], s[4:5], v32, v20, v[9:10]
-; SDAG-NEXT: v_add_i32_e32 v11, vcc, v14, v15
+; SDAG-NEXT: v_add_i32_e32 v11, vcc, v15, v14
; SDAG-NEXT: v_add_i32_e32 v8, vcc, v9, v8
; SDAG-NEXT: v_addc_u32_e32 v9, vcc, v10, v11, vcc
-; SDAG-NEXT: v_sub_i32_e32 v4, vcc, v4, v16
+; SDAG-NEXT: v_sub_i32_e32 v4, vcc, v4, v22
; SDAG-NEXT: v_subb_u32_e32 v5, vcc, v5, v12, vcc
; SDAG-NEXT: v_subb_u32_e32 v6, vcc, v6, v8, vcc
; SDAG-NEXT: v_subb_u32_e32 v7, vcc, v7, v9, vcc
diff --git a/llvm/test/CodeGen/AMDGPU/fceil64.ll b/llvm/test/CodeGen/AMDGPU/fceil64.ll
index 1c03167b42b9c..1e393a69774d8 100644
--- a/llvm/test/CodeGen/AMDGPU/fceil64.ll
+++ b/llvm/test/CodeGen/AMDGPU/fceil64.ll
@@ -407,7 +407,7 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: s_mov_b32 s2, -1
; SI-NEXT: s_mov_b32 s1, 0xfffff
; SI-NEXT: s_mov_b32 s0, s2
-; SI-NEXT: v_mov_b32_e32 v16, 0
+; SI-NEXT: v_mov_b32_e32 v12, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_bfe_u32 s3, s11, 0xb0014
; SI-NEXT: s_addk_i32 s3, 0xfc01
@@ -443,7 +443,7 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: v_mov_b32_e32 v1, s7
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[8:9], v[0:1]
; SI-NEXT: v_cmp_gt_f64_e64 s[8:9], s[8:9], 0
-; SI-NEXT: v_mov_b32_e32 v17, s3
+; SI-NEXT: v_mov_b32_e32 v13, s3
; SI-NEXT: s_and_b64 s[8:9], s[8:9], vcc
; SI-NEXT: s_and_b64 s[8:9], s[8:9], exec
; SI-NEXT: s_cselect_b32 s10, 0x3ff00000, 0
@@ -462,7 +462,7 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: v_mov_b32_e32 v1, s25
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[14:15], v[0:1]
; SI-NEXT: v_cmp_gt_f64_e64 s[8:9], s[14:15], 0
-; SI-NEXT: v_add_f64 v[2:3], s[26:27], v[16:17]
+; SI-NEXT: v_add_f64 v[2:3], s[26:27], v[12:13]
; SI-NEXT: s_and_b64 s[8:9], s[8:9], vcc
; SI-NEXT: s_and_b64 s[8:9], s[8:9], exec
; SI-NEXT: s_cselect_b32 s3, 0x3ff00000, 0
@@ -481,7 +481,7 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: v_mov_b32_e32 v1, s35
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[12:13], v[0:1]
; SI-NEXT: v_cmp_gt_f64_e64 s[8:9], s[12:13], 0
-; SI-NEXT: v_mov_b32_e32 v17, s10
+; SI-NEXT: v_mov_b32_e32 v13, s10
; SI-NEXT: s_and_b64 s[8:9], s[8:9], vcc
; SI-NEXT: s_and_b64 s[8:9], s[8:9], exec
; SI-NEXT: s_cselect_b32 s33, 0x3ff00000, 0
@@ -512,13 +512,13 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: s_cselect_b32 s8, 0, s8
; SI-NEXT: s_cselect_b32 s9, s11, s9
; SI-NEXT: s_cmp_gt_i32 s10, 51
-; SI-NEXT: s_cselect_b32 s29, s17, s9
-; SI-NEXT: s_cselect_b32 s28, s16, s8
-; SI-NEXT: v_mov_b32_e32 v0, s28
-; SI-NEXT: v_mov_b32_e32 v1, s29
+; SI-NEXT: s_cselect_b32 s27, s17, s9
+; SI-NEXT: s_cselect_b32 s26, s16, s8
+; SI-NEXT: v_mov_b32_e32 v0, s26
+; SI-NEXT: v_mov_b32_e32 v1, s27
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[16:17], v[0:1]
; SI-NEXT: v_cmp_gt_f64_e64 s[8:9], s[16:17], 0
-; SI-NEXT: v_add_f64 v[0:1], s[6:7], v[16:17]
+; SI-NEXT: v_add_f64 v[0:1], s[6:7], v[12:13]
; SI-NEXT: s_and_b64 s[6:7], s[8:9], vcc
; SI-NEXT: s_and_b64 s[6:7], s[6:7], exec
; SI-NEXT: s_cselect_b32 s39, 0x3ff00000, 0
@@ -531,13 +531,13 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: s_cselect_b32 s6, 0, s6
; SI-NEXT: s_cselect_b32 s7, s9, s7
; SI-NEXT: s_cmp_gt_i32 s8, 51
-; SI-NEXT: s_cselect_b32 s27, s23, s7
-; SI-NEXT: s_cselect_b32 s26, s22, s6
-; SI-NEXT: v_mov_b32_e32 v4, s26
-; SI-NEXT: v_mov_b32_e32 v5, s27
+; SI-NEXT: s_cselect_b32 s29, s23, s7
+; SI-NEXT: s_cselect_b32 s28, s22, s6
+; SI-NEXT: v_mov_b32_e32 v4, s28
+; SI-NEXT: v_mov_b32_e32 v5, s29
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[22:23], v[4:5]
; SI-NEXT: v_cmp_gt_f64_e64 s[6:7], s[22:23], 0
-; SI-NEXT: v_mov_b32_e32 v17, s3
+; SI-NEXT: v_mov_b32_e32 v13, s3
; SI-NEXT: s_and_b64 s[6:7], s[6:7], vcc
; SI-NEXT: s_and_b64 s[6:7], s[6:7], exec
; SI-NEXT: s_cselect_b32 s3, 0x3ff00000, 0
@@ -556,144 +556,144 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: v_mov_b32_e32 v5, s7
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[20:21], v[4:5]
; SI-NEXT: s_load_dwordx16 s[8:23], s[4:5], 0x39
-; SI-NEXT: v_add_f64 v[6:7], s[24:25], v[16:17]
+; SI-NEXT: v_add_f64 v[6:7], s[24:25], v[12:13]
; SI-NEXT: s_and_b64 s[24:25], s[36:37], vcc
; SI-NEXT: s_and_b64 s[24:25], s[24:25], exec
-; SI-NEXT: v_mov_b32_e32 v17, s33
+; SI-NEXT: v_mov_b32_e32 v13, s33
; SI-NEXT: s_cselect_b32 s33, 0x3ff00000, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_bfe_u32 s24, s11, 0xb0014
-; SI-NEXT: v_add_f64 v[4:5], s[34:35], v[16:17]
-; SI-NEXT: s_add_i32 s34, s24, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[24:25], s[0:1], s34
+; SI-NEXT: s_add_i32 s36, s24, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[24:25], s[0:1], s36
; SI-NEXT: s_andn2_b64 s[24:25], s[10:11], s[24:25]
-; SI-NEXT: s_and_b32 s35, s11, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s34, 0
+; SI-NEXT: s_and_b32 s37, s11, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s36, 0
; SI-NEXT: s_cselect_b32 s24, 0, s24
-; SI-NEXT: s_cselect_b32 s25, s35, s25
-; SI-NEXT: s_cmp_gt_i32 s34, 51
+; SI-NEXT: s_cselect_b32 s25, s37, s25
+; SI-NEXT: s_cmp_gt_i32 s36, 51
; SI-NEXT: s_cselect_b32 s25, s11, s25
; SI-NEXT: s_cselect_b32 s24, s10, s24
-; SI-NEXT: v_mov_b32_e32 v8, s24
-; SI-NEXT: v_mov_b32_e32 v9, s25
-; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[10:11], v[8:9]
+; SI-NEXT: v_mov_b32_e32 v4, s24
+; SI-NEXT: v_mov_b32_e32 v5, s25
+; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[10:11], v[4:5]
; SI-NEXT: v_cmp_gt_f64_e64 s[10:11], s[10:11], 0
-; SI-NEXT: v_mov_b32_e32 v17, s38
+; SI-NEXT: v_add_f64 v[4:5], s[34:35], v[12:13]
; SI-NEXT: s_and_b64 s[10:11], s[10:11], vcc
; SI-NEXT: s_and_b64 s[10:11], s[10:11], exec
-; SI-NEXT: v_add_f64 v[10:11], s[30:31], v[16:17]
-; SI-NEXT: s_cselect_b32 s30, 0x3ff00000, 0
+; SI-NEXT: s_cselect_b32 s34, 0x3ff00000, 0
; SI-NEXT: s_bfe_u32 s10, s9, 0xb0014
-; SI-NEXT: s_add_i32 s31, s10, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[10:11], s[0:1], s31
+; SI-NEXT: s_add_i32 s35, s10, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[10:11], s[0:1], s35
; SI-NEXT: s_andn2_b64 s[10:11], s[8:9], s[10:11]
-; SI-NEXT: s_and_b32 s34, s9, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s31, 0
+; SI-NEXT: s_and_b32 s36, s9, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s35, 0
; SI-NEXT: s_cselect_b32 s10, 0, s10
-; SI-NEXT: s_cselect_b32 s11, s34, s11
-; SI-NEXT: s_cmp_gt_i32 s31, 51
+; SI-NEXT: s_cselect_b32 s11, s36, s11
+; SI-NEXT: s_cmp_gt_i32 s35, 51
; SI-NEXT: s_cselect_b32 s11, s9, s11
; SI-NEXT: s_cselect_b32 s10, s8, s10
; SI-NEXT: v_mov_b32_e32 v8, s10
; SI-NEXT: v_mov_b32_e32 v9, s11
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[8:9], v[8:9]
; SI-NEXT: v_cmp_gt_f64_e64 s[8:9], s[8:9], 0
-; SI-NEXT: v_mov_b32_e32 v17, s39
+; SI-NEXT: v_mov_b32_e32 v13, s38
; SI-NEXT: s_and_b64 s[8:9], s[8:9], vcc
; SI-NEXT: s_and_b64 s[8:9], s[8:9], exec
-; SI-NEXT: s_cselect_b32 s31, 0x3ff00000, 0
+; SI-NEXT: s_cselect_b32 s35, 0x3ff00000, 0
; SI-NEXT: s_bfe_u32 s8, s15, 0xb0014
-; SI-NEXT: s_add_i32 s34, s8, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[8:9], s[0:1], s34
+; SI-NEXT: s_add_i32 s36, s8, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[8:9], s[0:1], s36
; SI-NEXT: s_andn2_b64 s[8:9], s[14:15], s[8:9]
-; SI-NEXT: s_and_b32 s35, s15, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s34, 0
+; SI-NEXT: s_and_b32 s37, s15, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s36, 0
; SI-NEXT: s_cselect_b32 s8, 0, s8
-; SI-NEXT: s_cselect_b32 s9, s35, s9
-; SI-NEXT: s_cmp_gt_i32 s34, 51
+; SI-NEXT: s_cselect_b32 s9, s37, s9
+; SI-NEXT: s_cmp_gt_i32 s36, 51
; SI-NEXT: s_cselect_b32 s9, s15, s9
; SI-NEXT: s_cselect_b32 s8, s14, s8
; SI-NEXT: v_mov_b32_e32 v8, s8
; SI-NEXT: v_mov_b32_e32 v9, s9
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[14:15], v[8:9]
; SI-NEXT: v_cmp_gt_f64_e64 s[14:15], s[14:15], 0
-; SI-NEXT: v_add_f64 v[8:9], s[28:29], v[16:17]
+; SI-NEXT: v_add_f64 v[10:11], s[30:31], v[12:13]
; SI-NEXT: s_and_b64 s[14:15], s[14:15], vcc
; SI-NEXT: s_and_b64 s[14:15], s[14:15], exec
-; SI-NEXT: s_cselect_b32 s28, 0x3ff00000, 0
+; SI-NEXT: s_cselect_b32 s30, 0x3ff00000, 0
; SI-NEXT: s_bfe_u32 s14, s13, 0xb0014
-; SI-NEXT: s_add_i32 s29, s14, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[14:15], s[0:1], s29
+; SI-NEXT: s_add_i32 s31, s14, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[14:15], s[0:1], s31
; SI-NEXT: s_andn2_b64 s[14:15], s[12:13], s[14:15]
-; SI-NEXT: s_and_b32 s34, s13, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s29, 0
+; SI-NEXT: s_and_b32 s36, s13, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s31, 0
; SI-NEXT: s_cselect_b32 s14, 0, s14
-; SI-NEXT: s_cselect_b32 s15, s34, s15
-; SI-NEXT: s_cmp_gt_i32 s29, 51
+; SI-NEXT: s_cselect_b32 s15, s36, s15
+; SI-NEXT: s_cmp_gt_i32 s31, 51
; SI-NEXT: s_cselect_b32 s15, s13, s15
; SI-NEXT: s_cselect_b32 s14, s12, s14
-; SI-NEXT: v_mov_b32_e32 v12, s14
-; SI-NEXT: v_mov_b32_e32 v13, s15
-; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[12:13], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v8, s14
+; SI-NEXT: v_mov_b32_e32 v9, s15
+; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[12:13], v[8:9]
; SI-NEXT: v_cmp_gt_f64_e64 s[12:13], s[12:13], 0
-; SI-NEXT: v_mov_b32_e32 v17, s3
+; SI-NEXT: v_mov_b32_e32 v13, s39
; SI-NEXT: s_and_b64 s[12:13], s[12:13], vcc
; SI-NEXT: s_and_b64 s[12:13], s[12:13], exec
-; SI-NEXT: s_cselect_b32 s3, 0x3ff00000, 0
+; SI-NEXT: s_cselect_b32 s31, 0x3ff00000, 0
; SI-NEXT: s_bfe_u32 s12, s19, 0xb0014
-; SI-NEXT: s_add_i32 s29, s12, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[12:13], s[0:1], s29
+; SI-NEXT: s_add_i32 s36, s12, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[12:13], s[0:1], s36
; SI-NEXT: s_andn2_b64 s[12:13], s[18:19], s[12:13]
-; SI-NEXT: s_and_b32 s34, s19, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s29, 0
+; SI-NEXT: s_and_b32 s37, s19, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s36, 0
+; SI-NEXT: s_cselect_b32 s13, s37, s13
; SI-NEXT: s_cselect_b32 s12, 0, s12
-; SI-NEXT: s_cselect_b32 s13, s34, s13
-; SI-NEXT: s_cmp_gt_i32 s29, 51
+; SI-NEXT: s_cmp_gt_i32 s36, 51
; SI-NEXT: s_cselect_b32 s13, s19, s13
; SI-NEXT: s_cselect_b32 s12, s18, s12
-; SI-NEXT: v_mov_b32_e32 v12, s12
-; SI-NEXT: v_mov_b32_e32 v13, s13
-; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[18:19], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v8, s12
+; SI-NEXT: v_mov_b32_e32 v9, s13
+; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[18:19], v[8:9]
; SI-NEXT: v_cmp_gt_f64_e64 s[18:19], s[18:19], 0
-; SI-NEXT: v_add_f64 v[14:15], s[26:27], v[16:17]
+; SI-NEXT: v_add_f64 v[8:9], s[26:27], v[12:13]
; SI-NEXT: s_and_b64 s[18:19], s[18:19], vcc
; SI-NEXT: s_and_b64 s[18:19], s[18:19], exec
-; SI-NEXT: s_cselect_b32 s26, 0x3ff00000, 0
+; SI-NEXT: v_mov_b32_e32 v13, s3
+; SI-NEXT: s_cselect_b32 s3, 0x3ff00000, 0
; SI-NEXT: s_bfe_u32 s18, s17, 0xb0014
-; SI-NEXT: s_add_i32 s27, s18, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[18:19], s[0:1], s27
+; SI-NEXT: s_add_i32 s26, s18, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[18:19], s[0:1], s26
; SI-NEXT: s_andn2_b64 s[18:19], s[16:17], s[18:19]
-; SI-NEXT: s_and_b32 s29, s17, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s27, 0
+; SI-NEXT: s_and_b32 s27, s17, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s26, 0
+; SI-NEXT: s_cselect_b32 s19, s27, s19
; SI-NEXT: s_cselect_b32 s18, 0, s18
-; SI-NEXT: s_cselect_b32 s19, s29, s19
-; SI-NEXT: s_cmp_gt_i32 s27, 51
+; SI-NEXT: s_cmp_gt_i32 s26, 51
; SI-NEXT: s_cselect_b32 s19, s17, s19
; SI-NEXT: s_cselect_b32 s18, s16, s18
-; SI-NEXT: v_mov_b32_e32 v12, s18
-; SI-NEXT: v_mov_b32_e32 v13, s19
-; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[16:17], v[12:13]
+; SI-NEXT: v_add_f64 v[16:17], s[28:29], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s18
+; SI-NEXT: v_mov_b32_e32 v14, s19
+; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[16:17], v[13:14]
; SI-NEXT: v_cmp_gt_f64_e64 s[16:17], s[16:17], 0
-; SI-NEXT: v_mov_b32_e32 v17, s33
+; SI-NEXT: v_mov_b32_e32 v13, s33
; SI-NEXT: s_and_b64 s[16:17], s[16:17], vcc
; SI-NEXT: s_and_b64 s[16:17], s[16:17], exec
-; SI-NEXT: s_cselect_b32 s27, 0x3ff00000, 0
+; SI-NEXT: s_cselect_b32 s26, 0x3ff00000, 0
; SI-NEXT: s_bfe_u32 s16, s23, 0xb0014
-; SI-NEXT: s_add_i32 s29, s16, 0xfffffc01
-; SI-NEXT: s_lshr_b64 s[16:17], s[0:1], s29
+; SI-NEXT: s_add_i32 s27, s16, 0xfffffc01
+; SI-NEXT: s_lshr_b64 s[16:17], s[0:1], s27
; SI-NEXT: s_andn2_b64 s[16:17], s[22:23], s[16:17]
-; SI-NEXT: s_and_b32 s33, s23, 0x80000000
-; SI-NEXT: s_cmp_lt_i32 s29, 0
-; SI-NEXT: s_cselect_b32 s17, s33, s17
+; SI-NEXT: s_and_b32 s28, s23, 0x80000000
+; SI-NEXT: s_cmp_lt_i32 s27, 0
+; SI-NEXT: s_cselect_b32 s17, s28, s17
; SI-NEXT: s_cselect_b32 s16, 0, s16
-; SI-NEXT: s_cmp_gt_i32 s29, 51
+; SI-NEXT: s_cmp_gt_i32 s27, 51
; SI-NEXT: s_cselect_b32 s17, s23, s17
; SI-NEXT: s_cselect_b32 s16, s22, s16
-; SI-NEXT: v_mov_b32_e32 v12, s16
-; SI-NEXT: v_mov_b32_e32 v13, s17
-; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[22:23], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v14, s16
+; SI-NEXT: v_mov_b32_e32 v15, s17
+; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[22:23], v[14:15]
; SI-NEXT: v_cmp_gt_f64_e64 s[22:23], s[22:23], 0
-; SI-NEXT: v_add_f64 v[12:13], s[6:7], v[16:17]
+; SI-NEXT: v_add_f64 v[14:15], s[6:7], v[12:13]
; SI-NEXT: s_and_b64 s[6:7], s[22:23], vcc
; SI-NEXT: s_and_b64 s[6:7], s[6:7], exec
; SI-NEXT: s_cselect_b32 s22, 0x3ff00000, 0
@@ -706,31 +706,31 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: s_cselect_b32 s1, s7, s1
; SI-NEXT: s_cselect_b32 s0, 0, s0
; SI-NEXT: s_cmp_gt_i32 s6, 51
-; SI-NEXT: v_mov_b32_e32 v17, s30
+; SI-NEXT: v_mov_b32_e32 v13, s34
; SI-NEXT: s_cselect_b32 s1, s21, s1
; SI-NEXT: s_cselect_b32 s0, s20, s0
; SI-NEXT: v_mov_b32_e32 v19, s1
; SI-NEXT: v_mov_b32_e32 v18, s0
-; SI-NEXT: v_add_f64 v[20:21], s[24:25], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s31
+; SI-NEXT: v_add_f64 v[20:21], s[24:25], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s35
; SI-NEXT: v_cmp_lg_f64_e32 vcc, s[20:21], v[18:19]
-; SI-NEXT: v_add_f64 v[18:19], s[10:11], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s28
+; SI-NEXT: v_add_f64 v[18:19], s[10:11], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s30
; SI-NEXT: v_cmp_gt_f64_e64 s[6:7], s[20:21], 0
-; SI-NEXT: v_add_f64 v[24:25], s[8:9], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s3
-; SI-NEXT: v_add_f64 v[22:23], s[14:15], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s26
+; SI-NEXT: v_add_f64 v[24:25], s[8:9], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s31
+; SI-NEXT: v_add_f64 v[22:23], s[14:15], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s3
; SI-NEXT: s_and_b64 s[6:7], s[6:7], vcc
-; SI-NEXT: v_add_f64 v[28:29], s[12:13], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s27
+; SI-NEXT: v_add_f64 v[28:29], s[12:13], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s26
; SI-NEXT: s_and_b64 s[6:7], s[6:7], exec
-; SI-NEXT: v_add_f64 v[26:27], s[18:19], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s22
+; SI-NEXT: v_add_f64 v[26:27], s[18:19], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s22
; SI-NEXT: s_cselect_b32 s3, 0x3ff00000, 0
-; SI-NEXT: v_add_f64 v[32:33], s[16:17], v[16:17]
-; SI-NEXT: v_mov_b32_e32 v17, s3
-; SI-NEXT: v_add_f64 v[30:31], s[0:1], v[16:17]
+; SI-NEXT: v_add_f64 v[32:33], s[16:17], v[12:13]
+; SI-NEXT: v_mov_b32_e32 v13, s3
+; SI-NEXT: v_add_f64 v[30:31], s[0:1], v[12:13]
; SI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x9
; SI-NEXT: s_mov_b32 s3, 0xf000
; SI-NEXT: s_waitcnt lgkmcnt(0)
@@ -738,7 +738,7 @@ define amdgpu_kernel void @fceil_v16f64(ptr addrspace(1) %out, <16 x double> %x)
; SI-NEXT: buffer_store_dwordx4 v[26:29], off, s[0:3], 0 offset:96
; SI-NEXT: buffer_store_dwordx4 v[22:25], off, s[0:3], 0 offset:80
; SI-NEXT: buffer_store_dwordx4 v[18:21], off, s[0:3], 0 offset:64
-; SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:48
+; SI-NEXT: buffer_store_dwordx4 v[14:17], off, s[0:3], 0 offset:48
; SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:32
; SI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:16
; SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0
diff --git a/llvm/test/CodeGen/AMDGPU/fmaximum.ll b/llvm/test/CodeGen/AMDGPU/fmaximum.ll
index 213d6ffeb9f00..512ea3cf71a3d 100644
--- a/llvm/test/CodeGen/AMDGPU/fmaximum.ll
+++ b/llvm/test/CodeGen/AMDGPU/fmaximum.ll
@@ -296,58 +296,111 @@ define amdgpu_ps <4 x float> @test_fmaximum_v4f32(<4 x float> %a, <4 x float> %b
}
define amdgpu_ps <16 x float> @test_fmaximum_v16f32(<16 x float> %a, <16 x float> %b) {
-; GFX9-LABEL: test_fmaximum_v16f32:
-; GFX9: ; %bb.0:
-; GFX9-NEXT: v_max_f32_e32 v32, v1, v17
-; GFX9-NEXT: v_mov_b32_e32 v33, 0x7fc00000
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
-; GFX9-NEXT: v_max_f32_e32 v1, v0, v16
-; GFX9-NEXT: v_cmp_o_f32_e64 s[12:13], v0, v16
-; GFX9-NEXT: v_max_f32_e32 v17, v2, v18
-; GFX9-NEXT: v_cmp_o_f32_e64 s[0:1], v2, v18
-; GFX9-NEXT: v_max_f32_e32 v18, v3, v19
-; GFX9-NEXT: v_cmp_o_f32_e64 s[2:3], v3, v19
-; GFX9-NEXT: v_max_f32_e32 v19, v4, v20
-; GFX9-NEXT: v_cmp_o_f32_e64 s[4:5], v4, v20
-; GFX9-NEXT: v_max_f32_e32 v20, v5, v21
-; GFX9-NEXT: v_cmp_o_f32_e64 s[6:7], v5, v21
-; GFX9-NEXT: v_max_f32_e32 v21, v6, v22
-; GFX9-NEXT: v_cmp_o_f32_e64 s[8:9], v6, v22
-; GFX9-NEXT: v_max_f32_e32 v22, v7, v23
-; GFX9-NEXT: v_cmp_o_f32_e64 s[10:11], v7, v23
-; GFX9-NEXT: v_max_f32_e32 v23, v8, v24
-; GFX9-NEXT: v_cndmask_b32_e64 v0, v33, v1, s[12:13]
-; GFX9-NEXT: v_cndmask_b32_e32 v1, v33, v32, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
-; GFX9-NEXT: v_max_f32_e32 v34, v9, v25
-; GFX9-NEXT: v_cndmask_b32_e32 v8, v33, v23, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
-; GFX9-NEXT: v_max_f32_e32 v35, v10, v26
-; GFX9-NEXT: v_cndmask_b32_e32 v9, v33, v34, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
-; GFX9-NEXT: v_max_f32_e32 v36, v11, v27
-; GFX9-NEXT: v_cndmask_b32_e32 v10, v33, v35, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
-; GFX9-NEXT: v_max_f32_e32 v37, v12, v28
-; GFX9-NEXT: v_cndmask_b32_e32 v11, v33, v36, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
-; GFX9-NEXT: v_max_f32_e32 v16, v13, v29
-; GFX9-NEXT: v_cndmask_b32_e32 v12, v33, v37, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
-; GFX9-NEXT: v_cndmask_b32_e32 v13, v33, v16, vcc
-; GFX9-NEXT: v_max_f32_e32 v16, v14, v30
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
-; GFX9-NEXT: v_cndmask_b32_e32 v14, v33, v16, vcc
-; GFX9-NEXT: v_max_f32_e32 v16, v15, v31
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
-; GFX9-NEXT: v_cndmask_b32_e64 v2, v33, v17, s[0:1]
-; GFX9-NEXT: v_cndmask_b32_e64 v3, v33, v18, s[2:3]
-; GFX9-NEXT: v_cndmask_b32_e64 v4, v33, v19, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e64 v5, v33, v20, s[6:7]
-; GFX9-NEXT: v_cndmask_b32_e64 v6, v33, v21, s[8:9]
-; GFX9-NEXT: v_cndmask_b32_e64 v7, v33, v22, s[10:11]
-; GFX9-NEXT: v_cndmask_b32_e32 v15, v33, v16, vcc
-; GFX9-NEXT: ; return to shader part epilog
+; GFX9-SDAG-LABEL: test_fmaximum_v16f32:
+; GFX9-SDAG: ; %bb.0:
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
+; GFX9-SDAG-NEXT: v_max_f32_e32 v0, v0, v16
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[0:1], v1, v17
+; GFX9-SDAG-NEXT: v_max_f32_e32 v1, v1, v17
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[2:3], v2, v18
+; GFX9-SDAG-NEXT: v_max_f32_e32 v2, v2, v18
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[4:5], v3, v19
+; GFX9-SDAG-NEXT: v_max_f32_e32 v3, v3, v19
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[6:7], v4, v20
+; GFX9-SDAG-NEXT: v_max_f32_e32 v4, v4, v20
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[8:9], v5, v21
+; GFX9-SDAG-NEXT: v_max_f32_e32 v5, v5, v21
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[10:11], v6, v22
+; GFX9-SDAG-NEXT: v_max_f32_e32 v6, v6, v22
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[12:13], v7, v23
+; GFX9-SDAG-NEXT: v_max_f32_e32 v7, v7, v23
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[14:15], v8, v24
+; GFX9-SDAG-NEXT: v_max_f32_e32 v8, v8, v24
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[16:17], v9, v25
+; GFX9-SDAG-NEXT: v_max_f32_e32 v9, v9, v25
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[20:21], v10, v26
+; GFX9-SDAG-NEXT: v_max_f32_e32 v10, v10, v26
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[22:23], v11, v27
+; GFX9-SDAG-NEXT: v_max_f32_e32 v11, v11, v27
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[24:25], v12, v28
+; GFX9-SDAG-NEXT: v_max_f32_e32 v12, v12, v28
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[26:27], v13, v29
+; GFX9-SDAG-NEXT: v_max_f32_e32 v13, v13, v29
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[28:29], v14, v30
+; GFX9-SDAG-NEXT: v_max_f32_e32 v14, v14, v30
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[18:19], v15, v31
+; GFX9-SDAG-NEXT: v_max_f32_e32 v15, v15, v31
+; GFX9-SDAG-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX9-SDAG-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v1, v16, v1, s[0:1]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v2, v16, v2, s[2:3]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v3, v16, v3, s[4:5]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v4, v16, v4, s[6:7]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v5, v16, v5, s[8:9]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v6, v16, v6, s[10:11]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v7, v16, v7, s[12:13]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v8, v16, v8, s[14:15]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v9, v16, v9, s[16:17]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v10, v16, v10, s[20:21]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v11, v16, v11, s[22:23]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v12, v16, v12, s[24:25]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v13, v16, v13, s[26:27]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v14, v16, v14, s[28:29]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v15, v16, v15, s[18:19]
+; GFX9-SDAG-NEXT: ; return to shader part epilog
+;
+; GFX9-GISEL-LABEL: test_fmaximum_v16f32:
+; GFX9-GISEL: ; %bb.0:
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
+; GFX9-GISEL-NEXT: v_max_f32_e32 v1, v1, v17
+; GFX9-GISEL-NEXT: v_mov_b32_e32 v17, 0x7fc00000
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[0:1], v2, v18
+; GFX9-GISEL-NEXT: v_max_f32_e32 v2, v2, v18
+; GFX9-GISEL-NEXT: v_max_f32_e32 v18, v0, v16
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[24:25], v0, v16
+; GFX9-GISEL-NEXT: v_cndmask_b32_e32 v1, v17, v1, vcc
+; GFX9-GISEL-NEXT: v_max_f32_e32 v16, v14, v30
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[2:3], v3, v19
+; GFX9-GISEL-NEXT: v_max_f32_e32 v3, v3, v19
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[4:5], v4, v20
+; GFX9-GISEL-NEXT: v_max_f32_e32 v4, v4, v20
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[6:7], v5, v21
+; GFX9-GISEL-NEXT: v_max_f32_e32 v5, v5, v21
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[8:9], v6, v22
+; GFX9-GISEL-NEXT: v_max_f32_e32 v6, v6, v22
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[10:11], v7, v23
+; GFX9-GISEL-NEXT: v_max_f32_e32 v7, v7, v23
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[12:13], v8, v24
+; GFX9-GISEL-NEXT: v_max_f32_e32 v8, v8, v24
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[14:15], v9, v25
+; GFX9-GISEL-NEXT: v_max_f32_e32 v9, v9, v25
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[16:17], v10, v26
+; GFX9-GISEL-NEXT: v_max_f32_e32 v10, v10, v26
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[18:19], v11, v27
+; GFX9-GISEL-NEXT: v_max_f32_e32 v11, v11, v27
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[20:21], v12, v28
+; GFX9-GISEL-NEXT: v_max_f32_e32 v12, v12, v28
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[22:23], v13, v29
+; GFX9-GISEL-NEXT: v_max_f32_e32 v13, v13, v29
+; GFX9-GISEL-NEXT: v_cndmask_b32_e32 v14, v17, v16, vcc
+; GFX9-GISEL-NEXT: v_max_f32_e32 v16, v15, v31
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v0, v17, v18, s[24:25]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[0:1]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[2:3]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[4:5]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[6:7]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[8:9]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[10:11]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[12:13]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[14:15]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[16:17]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[18:19]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[20:21]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v13, v17, v13, s[22:23]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e32 v15, v17, v16, vcc
+; GFX9-GISEL-NEXT: ; return to shader part epilog
;
; GFX1170-LABEL: test_fmaximum_v16f32:
; GFX1170: ; %bb.0:
diff --git a/llvm/test/CodeGen/AMDGPU/fminimum.ll b/llvm/test/CodeGen/AMDGPU/fminimum.ll
index 094dd76e0f945..cad843fde2fcd 100644
--- a/llvm/test/CodeGen/AMDGPU/fminimum.ll
+++ b/llvm/test/CodeGen/AMDGPU/fminimum.ll
@@ -296,58 +296,111 @@ define amdgpu_ps <4 x float> @test_fminimum_v4f32(<4 x float> %a, <4 x float> %b
}
define amdgpu_ps <16 x float> @test_fminimum_v16f32(<16 x float> %a, <16 x float> %b) {
-; GFX9-LABEL: test_fminimum_v16f32:
-; GFX9: ; %bb.0:
-; GFX9-NEXT: v_min_f32_e32 v32, v1, v17
-; GFX9-NEXT: v_mov_b32_e32 v33, 0x7fc00000
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
-; GFX9-NEXT: v_min_f32_e32 v1, v0, v16
-; GFX9-NEXT: v_cmp_o_f32_e64 s[12:13], v0, v16
-; GFX9-NEXT: v_min_f32_e32 v17, v2, v18
-; GFX9-NEXT: v_cmp_o_f32_e64 s[0:1], v2, v18
-; GFX9-NEXT: v_min_f32_e32 v18, v3, v19
-; GFX9-NEXT: v_cmp_o_f32_e64 s[2:3], v3, v19
-; GFX9-NEXT: v_min_f32_e32 v19, v4, v20
-; GFX9-NEXT: v_cmp_o_f32_e64 s[4:5], v4, v20
-; GFX9-NEXT: v_min_f32_e32 v20, v5, v21
-; GFX9-NEXT: v_cmp_o_f32_e64 s[6:7], v5, v21
-; GFX9-NEXT: v_min_f32_e32 v21, v6, v22
-; GFX9-NEXT: v_cmp_o_f32_e64 s[8:9], v6, v22
-; GFX9-NEXT: v_min_f32_e32 v22, v7, v23
-; GFX9-NEXT: v_cmp_o_f32_e64 s[10:11], v7, v23
-; GFX9-NEXT: v_min_f32_e32 v23, v8, v24
-; GFX9-NEXT: v_cndmask_b32_e64 v0, v33, v1, s[12:13]
-; GFX9-NEXT: v_cndmask_b32_e32 v1, v33, v32, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
-; GFX9-NEXT: v_min_f32_e32 v34, v9, v25
-; GFX9-NEXT: v_cndmask_b32_e32 v8, v33, v23, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
-; GFX9-NEXT: v_min_f32_e32 v35, v10, v26
-; GFX9-NEXT: v_cndmask_b32_e32 v9, v33, v34, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
-; GFX9-NEXT: v_min_f32_e32 v36, v11, v27
-; GFX9-NEXT: v_cndmask_b32_e32 v10, v33, v35, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
-; GFX9-NEXT: v_min_f32_e32 v37, v12, v28
-; GFX9-NEXT: v_cndmask_b32_e32 v11, v33, v36, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
-; GFX9-NEXT: v_min_f32_e32 v16, v13, v29
-; GFX9-NEXT: v_cndmask_b32_e32 v12, v33, v37, vcc
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
-; GFX9-NEXT: v_cndmask_b32_e32 v13, v33, v16, vcc
-; GFX9-NEXT: v_min_f32_e32 v16, v14, v30
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
-; GFX9-NEXT: v_cndmask_b32_e32 v14, v33, v16, vcc
-; GFX9-NEXT: v_min_f32_e32 v16, v15, v31
-; GFX9-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
-; GFX9-NEXT: v_cndmask_b32_e64 v2, v33, v17, s[0:1]
-; GFX9-NEXT: v_cndmask_b32_e64 v3, v33, v18, s[2:3]
-; GFX9-NEXT: v_cndmask_b32_e64 v4, v33, v19, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e64 v5, v33, v20, s[6:7]
-; GFX9-NEXT: v_cndmask_b32_e64 v6, v33, v21, s[8:9]
-; GFX9-NEXT: v_cndmask_b32_e64 v7, v33, v22, s[10:11]
-; GFX9-NEXT: v_cndmask_b32_e32 v15, v33, v16, vcc
-; GFX9-NEXT: ; return to shader part epilog
+; GFX9-SDAG-LABEL: test_fminimum_v16f32:
+; GFX9-SDAG: ; %bb.0:
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
+; GFX9-SDAG-NEXT: v_min_f32_e32 v0, v0, v16
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[0:1], v1, v17
+; GFX9-SDAG-NEXT: v_min_f32_e32 v1, v1, v17
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[2:3], v2, v18
+; GFX9-SDAG-NEXT: v_min_f32_e32 v2, v2, v18
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[4:5], v3, v19
+; GFX9-SDAG-NEXT: v_min_f32_e32 v3, v3, v19
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[6:7], v4, v20
+; GFX9-SDAG-NEXT: v_min_f32_e32 v4, v4, v20
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[8:9], v5, v21
+; GFX9-SDAG-NEXT: v_min_f32_e32 v5, v5, v21
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[10:11], v6, v22
+; GFX9-SDAG-NEXT: v_min_f32_e32 v6, v6, v22
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[12:13], v7, v23
+; GFX9-SDAG-NEXT: v_min_f32_e32 v7, v7, v23
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[14:15], v8, v24
+; GFX9-SDAG-NEXT: v_min_f32_e32 v8, v8, v24
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[16:17], v9, v25
+; GFX9-SDAG-NEXT: v_min_f32_e32 v9, v9, v25
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[20:21], v10, v26
+; GFX9-SDAG-NEXT: v_min_f32_e32 v10, v10, v26
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[22:23], v11, v27
+; GFX9-SDAG-NEXT: v_min_f32_e32 v11, v11, v27
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[24:25], v12, v28
+; GFX9-SDAG-NEXT: v_min_f32_e32 v12, v12, v28
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[26:27], v13, v29
+; GFX9-SDAG-NEXT: v_min_f32_e32 v13, v13, v29
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[28:29], v14, v30
+; GFX9-SDAG-NEXT: v_min_f32_e32 v14, v14, v30
+; GFX9-SDAG-NEXT: v_cmp_o_f32_e64 s[18:19], v15, v31
+; GFX9-SDAG-NEXT: v_min_f32_e32 v15, v15, v31
+; GFX9-SDAG-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX9-SDAG-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v1, v16, v1, s[0:1]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v2, v16, v2, s[2:3]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v3, v16, v3, s[4:5]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v4, v16, v4, s[6:7]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v5, v16, v5, s[8:9]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v6, v16, v6, s[10:11]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v7, v16, v7, s[12:13]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v8, v16, v8, s[14:15]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v9, v16, v9, s[16:17]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v10, v16, v10, s[20:21]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v11, v16, v11, s[22:23]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v12, v16, v12, s[24:25]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v13, v16, v13, s[26:27]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v14, v16, v14, s[28:29]
+; GFX9-SDAG-NEXT: v_cndmask_b32_e64 v15, v16, v15, s[18:19]
+; GFX9-SDAG-NEXT: ; return to shader part epilog
+;
+; GFX9-GISEL-LABEL: test_fminimum_v16f32:
+; GFX9-GISEL: ; %bb.0:
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
+; GFX9-GISEL-NEXT: v_min_f32_e32 v1, v1, v17
+; GFX9-GISEL-NEXT: v_mov_b32_e32 v17, 0x7fc00000
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[0:1], v2, v18
+; GFX9-GISEL-NEXT: v_min_f32_e32 v2, v2, v18
+; GFX9-GISEL-NEXT: v_min_f32_e32 v18, v0, v16
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[24:25], v0, v16
+; GFX9-GISEL-NEXT: v_cndmask_b32_e32 v1, v17, v1, vcc
+; GFX9-GISEL-NEXT: v_min_f32_e32 v16, v14, v30
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[2:3], v3, v19
+; GFX9-GISEL-NEXT: v_min_f32_e32 v3, v3, v19
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[4:5], v4, v20
+; GFX9-GISEL-NEXT: v_min_f32_e32 v4, v4, v20
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[6:7], v5, v21
+; GFX9-GISEL-NEXT: v_min_f32_e32 v5, v5, v21
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[8:9], v6, v22
+; GFX9-GISEL-NEXT: v_min_f32_e32 v6, v6, v22
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[10:11], v7, v23
+; GFX9-GISEL-NEXT: v_min_f32_e32 v7, v7, v23
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[12:13], v8, v24
+; GFX9-GISEL-NEXT: v_min_f32_e32 v8, v8, v24
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[14:15], v9, v25
+; GFX9-GISEL-NEXT: v_min_f32_e32 v9, v9, v25
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[16:17], v10, v26
+; GFX9-GISEL-NEXT: v_min_f32_e32 v10, v10, v26
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[18:19], v11, v27
+; GFX9-GISEL-NEXT: v_min_f32_e32 v11, v11, v27
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[20:21], v12, v28
+; GFX9-GISEL-NEXT: v_min_f32_e32 v12, v12, v28
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e64 s[22:23], v13, v29
+; GFX9-GISEL-NEXT: v_min_f32_e32 v13, v13, v29
+; GFX9-GISEL-NEXT: v_cndmask_b32_e32 v14, v17, v16, vcc
+; GFX9-GISEL-NEXT: v_min_f32_e32 v16, v15, v31
+; GFX9-GISEL-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v0, v17, v18, s[24:25]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[0:1]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[2:3]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[4:5]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[6:7]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[8:9]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[10:11]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[12:13]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[14:15]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[16:17]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[18:19]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[20:21]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e64 v13, v17, v13, s[22:23]
+; GFX9-GISEL-NEXT: v_cndmask_b32_e32 v15, v17, v16, vcc
+; GFX9-GISEL-NEXT: ; return to shader part epilog
;
; GFX1170-LABEL: test_fminimum_v16f32:
; GFX1170: ; %bb.0:
diff --git a/llvm/test/CodeGen/AMDGPU/freeze.ll b/llvm/test/CodeGen/AMDGPU/freeze.ll
index 5d520d8e88e14..2615b190ce78e 100644
--- a/llvm/test/CodeGen/AMDGPU/freeze.ll
+++ b/llvm/test/CodeGen/AMDGPU/freeze.ll
@@ -3550,37 +3550,36 @@ define void @freeze_v19i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GISEL-LABEL: freeze_v19i32:
; GFX8-GISEL: ; %bb.0:
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 16, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 32, v0
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 16, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 32, v0
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 48, v0
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
; GFX8-GISEL-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 64, v0
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GISEL-NEXT: flat_load_dwordx3 v[20:22], v[0:1]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
-; GFX8-GISEL-NEXT: flat_load_dwordx3 v[20:22], v[20:21]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 16, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v23, vcc, 16, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v24, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 48, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, 64, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[4:7]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[12:15]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[23:24], v[8:11]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 48, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v2, vcc, 64, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(3)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx3 v[6:7], v[20:22]
+; GFX8-GISEL-NEXT: flat_store_dwordx3 v[2:3], v[20:22]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX8-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -3817,36 +3816,34 @@ define void @freeze_v20i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX803-SDAG-LABEL: freeze_v20i32:
; GFX8-GFX803-SDAG: ; %bb.0:
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, 32, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 64, v0
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 32, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, 64, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v20, vcc, 16, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 48, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[16:19], v[0:1]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 64, v2
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, 64, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 32, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[4:7]
-; GFX8-GFX803-SDAG-NEXT: s_nop 0
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, 32, v2
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[24:25], v[4:7]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(3)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[12:15]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, 48, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, 48, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v2, vcc, 16, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(4)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[12:15]
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[6:7], v[16:19]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(4)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0)
@@ -3857,34 +3854,33 @@ define void @freeze_v20i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 16, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 32, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 48, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 64, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 16, v2
+; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, 16, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[4:7]
-; GFX8-GISEL-NEXT: s_nop 0
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 32, v2
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[24:25], v[8:11]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 48, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, 48, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: v_add_u32_e32 v2, vcc, 64, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[12:15]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[6:7], v[16:19]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
@@ -4030,20 +4026,20 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-SDAG-NEXT: s_mov_b32 s4, s6
; GFX6-SDAG-NEXT: s_mov_b32 s5, s6
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[4:7], v[0:1], s[4:7], 0 addr64 offset:64
+; GFX6-SDAG-NEXT: buffer_load_dword v24, v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:32
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:48
-; GFX6-SDAG-NEXT: buffer_load_dword v24, v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:16
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64 offset:64
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
+; GFX6-SDAG-NEXT: buffer_store_dword v24, v[2:3], s[4:7], 0 addr64 offset:80
+; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[8:11], v[2:3], s[4:7], 0 addr64 offset:32
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:48
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX6-SDAG-NEXT: buffer_store_dword v24, v[2:3], s[4:7], 0 addr64 offset:80
-; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:16
@@ -4060,8 +4056,8 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:16
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:32
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:48
+; GFX6-GISEL-NEXT: buffer_load_dword v24, v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:64
-; GFX6-GISEL-NEXT: buffer_load_dword v0, v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
@@ -4070,10 +4066,9 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:32
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64 offset:48
-; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
+; GFX6-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:64
-; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
-; GFX6-GISEL-NEXT: buffer_store_dword v0, v[2:3], s[4:7], 0 addr64 offset:80
+; GFX6-GISEL-NEXT: buffer_store_dword v24, v[2:3], s[4:7], 0 addr64 offset:80
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0)
; GFX6-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4085,20 +4080,20 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-SDAG-NEXT: s_mov_b32 s4, s6
; GFX7-SDAG-NEXT: s_mov_b32 s5, s6
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[4:7], v[0:1], s[4:7], 0 addr64 offset:64
+; GFX7-SDAG-NEXT: buffer_load_dword v24, v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:32
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:48
-; GFX7-SDAG-NEXT: buffer_load_dword v24, v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:16
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64 offset:64
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
+; GFX7-SDAG-NEXT: buffer_store_dword v24, v[2:3], s[4:7], 0 addr64 offset:80
+; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[8:11], v[2:3], s[4:7], 0 addr64 offset:32
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:48
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX7-SDAG-NEXT: buffer_store_dword v24, v[2:3], s[4:7], 0 addr64 offset:80
-; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:16
@@ -4115,8 +4110,8 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:16
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:32
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:48
+; GFX7-GISEL-NEXT: buffer_load_dword v24, v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:64
-; GFX7-GISEL-NEXT: buffer_load_dword v0, v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
@@ -4125,10 +4120,9 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:32
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64 offset:48
-; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
+; GFX7-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:64
-; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
-; GFX7-GISEL-NEXT: buffer_store_dword v0, v[2:3], s[4:7], 0 addr64 offset:80
+; GFX7-GISEL-NEXT: buffer_store_dword v24, v[2:3], s[4:7], 0 addr64 offset:80
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX7-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4224,20 +4218,20 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX900-SDAG: ; %bb.0:
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[4:7], v[0:1], off offset:64
+; GFX8-GFX900-SDAG-NEXT: global_load_dword v24, v[0:1], off offset:80
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[8:11], v[0:1], off offset:32
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[12:15], v[0:1], off offset:48
-; GFX8-GFX900-SDAG-NEXT: global_load_dword v24, v[0:1], off offset:80
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[16:19], v[0:1], off
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:16
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[4:7], off offset:64
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
+; GFX8-GFX900-SDAG-NEXT: global_store_dword v[2:3], v24, off offset:80
+; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[8:11], off offset:32
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[12:15], off offset:48
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX900-SDAG-NEXT: global_store_dword v[2:3], v24, off offset:80
-; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[16:19], off
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[20:23], off offset:16
@@ -4251,8 +4245,8 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX9-GISEL-NEXT: global_load_dwordx4 v[8:11], v[0:1], off offset:16
; GFX9-GISEL-NEXT: global_load_dwordx4 v[12:15], v[0:1], off offset:32
; GFX9-GISEL-NEXT: global_load_dwordx4 v[16:19], v[0:1], off offset:48
-; GFX9-GISEL-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:64
; GFX9-GISEL-NEXT: global_load_dword v24, v[0:1], off offset:80
+; GFX9-GISEL-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:64
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[4:7], off
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
@@ -4261,9 +4255,8 @@ define void @freeze_v21i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[12:15], off offset:32
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[16:19], off offset:48
-; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
+; GFX9-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[20:23], off offset:64
-; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX9-GISEL-NEXT: global_store_dword v[2:3], v24, off offset:80
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX9-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -4378,20 +4371,20 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-SDAG-NEXT: s_mov_b32 s4, s6
; GFX6-SDAG-NEXT: s_mov_b32 s5, s6
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[4:7], v[0:1], s[4:7], 0 addr64 offset:64
+; GFX6-SDAG-NEXT: buffer_load_dwordx2 v[24:25], v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:32
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:48
-; GFX6-SDAG-NEXT: buffer_load_dwordx2 v[24:25], v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:16
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64 offset:64
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
+; GFX6-SDAG-NEXT: buffer_store_dwordx2 v[24:25], v[2:3], s[4:7], 0 addr64 offset:80
+; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[8:11], v[2:3], s[4:7], 0 addr64 offset:32
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:48
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX6-SDAG-NEXT: buffer_store_dwordx2 v[24:25], v[2:3], s[4:7], 0 addr64 offset:80
-; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:16
@@ -4408,8 +4401,8 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:16
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:32
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:48
+; GFX6-GISEL-NEXT: buffer_load_dwordx2 v[24:25], v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:64
-; GFX6-GISEL-NEXT: buffer_load_dwordx2 v[0:1], v[0:1], s[4:7], 0 addr64 offset:80
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
@@ -4418,10 +4411,9 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:32
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64 offset:48
-; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
+; GFX6-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:64
-; GFX6-GISEL-NEXT: s_waitcnt vmcnt(5)
-; GFX6-GISEL-NEXT: buffer_store_dwordx2 v[0:1], v[2:3], s[4:7], 0 addr64 offset:80
+; GFX6-GISEL-NEXT: buffer_store_dwordx2 v[24:25], v[2:3], s[4:7], 0 addr64 offset:80
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0)
; GFX6-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4433,20 +4425,20 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-SDAG-NEXT: s_mov_b32 s4, s6
; GFX7-SDAG-NEXT: s_mov_b32 s5, s6
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[4:7], v[0:1], s[4:7], 0 addr64 offset:64
+; GFX7-SDAG-NEXT: buffer_load_dwordx2 v[24:25], v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:32
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:48
-; GFX7-SDAG-NEXT: buffer_load_dwordx2 v[24:25], v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:16
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64 offset:64
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
+; GFX7-SDAG-NEXT: buffer_store_dwordx2 v[24:25], v[2:3], s[4:7], 0 addr64 offset:80
+; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[8:11], v[2:3], s[4:7], 0 addr64 offset:32
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:48
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX7-SDAG-NEXT: buffer_store_dwordx2 v[24:25], v[2:3], s[4:7], 0 addr64 offset:80
-; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:16
@@ -4463,8 +4455,8 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:16
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:32
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:48
+; GFX7-GISEL-NEXT: buffer_load_dwordx2 v[24:25], v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:64
-; GFX7-GISEL-NEXT: buffer_load_dwordx2 v[0:1], v[0:1], s[4:7], 0 addr64 offset:80
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
@@ -4473,10 +4465,9 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:32
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64 offset:48
-; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
+; GFX7-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:64
-; GFX7-GISEL-NEXT: s_waitcnt vmcnt(5)
-; GFX7-GISEL-NEXT: buffer_store_dwordx2 v[0:1], v[2:3], s[4:7], 0 addr64 offset:80
+; GFX7-GISEL-NEXT: buffer_store_dwordx2 v[24:25], v[2:3], s[4:7], 0 addr64 offset:80
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX7-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4571,20 +4562,20 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX900-SDAG: ; %bb.0:
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[4:7], v[0:1], off offset:64
+; GFX8-GFX900-SDAG-NEXT: global_load_dwordx2 v[24:25], v[0:1], off offset:80
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[8:11], v[0:1], off offset:32
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[12:15], v[0:1], off offset:48
-; GFX8-GFX900-SDAG-NEXT: global_load_dwordx2 v[24:25], v[0:1], off offset:80
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[16:19], v[0:1], off
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:16
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[4:7], off offset:64
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
+; GFX8-GFX900-SDAG-NEXT: global_store_dwordx2 v[2:3], v[24:25], off offset:80
+; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[8:11], off offset:32
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[12:15], off offset:48
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX900-SDAG-NEXT: global_store_dwordx2 v[2:3], v[24:25], off offset:80
-; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[16:19], off
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(5)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[20:23], off offset:16
@@ -4598,8 +4589,8 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX9-GISEL-NEXT: global_load_dwordx4 v[8:11], v[0:1], off offset:16
; GFX9-GISEL-NEXT: global_load_dwordx4 v[12:15], v[0:1], off offset:32
; GFX9-GISEL-NEXT: global_load_dwordx4 v[16:19], v[0:1], off offset:48
-; GFX9-GISEL-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:64
; GFX9-GISEL-NEXT: global_load_dwordx2 v[24:25], v[0:1], off offset:80
+; GFX9-GISEL-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:64
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[4:7], off
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
@@ -4608,9 +4599,8 @@ define void @freeze_v22i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[12:15], off offset:32
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[16:19], off offset:48
-; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
+; GFX9-GISEL-NEXT: s_waitcnt vmcnt(4)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[20:23], off offset:64
-; GFX9-GISEL-NEXT: s_waitcnt vmcnt(5)
; GFX9-GISEL-NEXT: global_store_dwordx2 v[2:3], v[24:25], off offset:80
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX9-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -4727,8 +4717,8 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[4:7], v[0:1], s[4:7], 0 addr64 offset:96
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:64
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:80
-; GFX6-SDAG-NEXT: buffer_load_dwordx2 v[32:33], v[0:1], s[4:7], 0 addr64 offset:112
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:32
+; GFX6-SDAG-NEXT: buffer_load_dwordx2 v[32:33], v[0:1], s[4:7], 0 addr64 offset:112
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:48
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[24:27], v[0:1], s[4:7], 0 addr64
; GFX6-SDAG-NEXT: buffer_load_dwordx4 v[28:31], v[0:1], s[4:7], 0 addr64 offset:16
@@ -4739,10 +4729,10 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:80
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX6-SDAG-NEXT: buffer_store_dwordx2 v[32:33], v[2:3], s[4:7], 0 addr64 offset:112
-; GFX6-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64 offset:32
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX6-SDAG-NEXT: buffer_store_dwordx2 v[32:33], v[2:3], s[4:7], 0 addr64 offset:112
+; GFX6-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:48
; GFX6-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX6-SDAG-NEXT: buffer_store_dwordx4 v[24:27], v[2:3], s[4:7], 0 addr64
@@ -4763,8 +4753,8 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:48
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:64
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[24:27], v[0:1], s[4:7], 0 addr64 offset:80
+; GFX6-GISEL-NEXT: buffer_load_dwordx2 v[32:33], v[0:1], s[4:7], 0 addr64 offset:112
; GFX6-GISEL-NEXT: buffer_load_dwordx4 v[28:31], v[0:1], s[4:7], 0 addr64 offset:96
-; GFX6-GISEL-NEXT: buffer_load_dwordx2 v[0:1], v[0:1], s[4:7], 0 addr64 offset:112
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(7)
@@ -4777,10 +4767,9 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:64
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[24:27], v[2:3], s[4:7], 0 addr64 offset:80
-; GFX6-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX6-GISEL-NEXT: s_waitcnt vmcnt(6)
; GFX6-GISEL-NEXT: buffer_store_dwordx4 v[28:31], v[2:3], s[4:7], 0 addr64 offset:96
-; GFX6-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX6-GISEL-NEXT: buffer_store_dwordx2 v[0:1], v[2:3], s[4:7], 0 addr64 offset:112
+; GFX6-GISEL-NEXT: buffer_store_dwordx2 v[32:33], v[2:3], s[4:7], 0 addr64 offset:112
; GFX6-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0)
; GFX6-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4794,8 +4783,8 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[4:7], v[0:1], s[4:7], 0 addr64 offset:96
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[8:11], v[0:1], s[4:7], 0 addr64 offset:64
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[12:15], v[0:1], s[4:7], 0 addr64 offset:80
-; GFX7-SDAG-NEXT: buffer_load_dwordx2 v[32:33], v[0:1], s[4:7], 0 addr64 offset:112
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:32
+; GFX7-SDAG-NEXT: buffer_load_dwordx2 v[32:33], v[0:1], s[4:7], 0 addr64 offset:112
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:48
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[24:27], v[0:1], s[4:7], 0 addr64
; GFX7-SDAG-NEXT: buffer_load_dwordx4 v[28:31], v[0:1], s[4:7], 0 addr64 offset:16
@@ -4806,10 +4795,10 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[12:15], v[2:3], s[4:7], 0 addr64 offset:80
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX7-SDAG-NEXT: buffer_store_dwordx2 v[32:33], v[2:3], s[4:7], 0 addr64 offset:112
-; GFX7-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[16:19], v[2:3], s[4:7], 0 addr64 offset:32
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX7-SDAG-NEXT: buffer_store_dwordx2 v[32:33], v[2:3], s[4:7], 0 addr64 offset:112
+; GFX7-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:48
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX7-SDAG-NEXT: buffer_store_dwordx4 v[24:27], v[2:3], s[4:7], 0 addr64
@@ -4830,8 +4819,8 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[16:19], v[0:1], s[4:7], 0 addr64 offset:48
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[20:23], v[0:1], s[4:7], 0 addr64 offset:64
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[24:27], v[0:1], s[4:7], 0 addr64 offset:80
+; GFX7-GISEL-NEXT: buffer_load_dwordx2 v[32:33], v[0:1], s[4:7], 0 addr64 offset:112
; GFX7-GISEL-NEXT: buffer_load_dwordx4 v[28:31], v[0:1], s[4:7], 0 addr64 offset:96
-; GFX7-GISEL-NEXT: buffer_load_dwordx2 v[0:1], v[0:1], s[4:7], 0 addr64 offset:112
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[4:7], v[2:3], s[4:7], 0 addr64
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(7)
@@ -4844,10 +4833,9 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[20:23], v[2:3], s[4:7], 0 addr64 offset:64
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[24:27], v[2:3], s[4:7], 0 addr64 offset:80
-; GFX7-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX7-GISEL-NEXT: s_waitcnt vmcnt(6)
; GFX7-GISEL-NEXT: buffer_store_dwordx4 v[28:31], v[2:3], s[4:7], 0 addr64 offset:96
-; GFX7-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX7-GISEL-NEXT: buffer_store_dwordx2 v[0:1], v[2:3], s[4:7], 0 addr64 offset:112
+; GFX7-GISEL-NEXT: buffer_store_dwordx2 v[32:33], v[2:3], s[4:7], 0 addr64 offset:112
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX7-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4915,61 +4903,60 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 16, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 32, v0
+; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 32, v0
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 48, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 48, v0
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[0:1]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 64, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v34, 0x50
-; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v34
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v36, 0x50
+; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v36
; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x60
; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v14
; GFX8-GISEL-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x70
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v0, v14
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v18, 0x70
+; GFX8-GISEL-NEXT: v_add_u32_e32 v18, vcc, v0, v18
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v19, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx2 v[32:33], v[18:19]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
-; GFX8-GISEL-NEXT: flat_load_dwordx2 v[0:1], v[0:1]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v32, vcc, 16, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v33, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v34, vcc, 16, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v35, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[32:33], v[4:7]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[34:35], v[4:7]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[8:11]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 32, v2
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 48, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 64, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, 48, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v32, vcc, v2, v34
-; GFX8-GISEL-NEXT: v_add_u32_e64 v34, s[4:5], 64, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v35, s[4:5], 0, v3, s[4:5]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, v2, v36
; GFX8-GISEL-NEXT: s_mov_b64 s[4:5], vcc
; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 0x60, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v33, s[4:5], 0, v3, s[4:5]
+; GFX8-GISEL-NEXT: v_addc_u32_e64 v7, s[4:5], 0, v3, s[4:5]
; GFX8-GISEL-NEXT: s_mov_b64 s[4:5], vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v2, vcc, 0x70, v2
; GFX8-GISEL-NEXT: v_addc_u32_e64 v9, s[4:5], 0, v3, s[4:5]
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[12:15]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[6:7], v[16:19]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[34:35], v[20:23]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 0x70, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[8:9], v[28:31]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx2 v[2:3], v[0:1]
+; GFX8-GISEL-NEXT: flat_store_dwordx2 v[0:1], v[32:33]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX8-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -4979,8 +4966,8 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[4:7], v[0:1], off offset:96
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[8:11], v[0:1], off offset:64
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[12:15], v[0:1], off offset:80
-; GFX8-GFX900-SDAG-NEXT: global_load_dwordx2 v[32:33], v[0:1], off offset:112
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[16:19], v[0:1], off offset:32
+; GFX8-GFX900-SDAG-NEXT: global_load_dwordx2 v[32:33], v[0:1], off offset:112
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:48
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[24:27], v[0:1], off
; GFX8-GFX900-SDAG-NEXT: global_load_dwordx4 v[28:31], v[0:1], off offset:16
@@ -4991,10 +4978,10 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[12:15], off offset:80
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX900-SDAG-NEXT: global_store_dwordx2 v[2:3], v[32:33], off offset:112
-; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[16:19], off offset:32
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX900-SDAG-NEXT: global_store_dwordx2 v[2:3], v[32:33], off offset:112
+; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[20:23], off offset:48
; GFX8-GFX900-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX900-SDAG-NEXT: global_store_dwordx4 v[2:3], v[24:27], off
@@ -5012,9 +4999,8 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX9-GISEL-NEXT: global_load_dwordx4 v[16:19], v[0:1], off offset:48
; GFX9-GISEL-NEXT: global_load_dwordx4 v[20:23], v[0:1], off offset:64
; GFX9-GISEL-NEXT: global_load_dwordx4 v[24:27], v[0:1], off offset:80
+; GFX9-GISEL-NEXT: global_load_dwordx2 v[32:33], v[0:1], off offset:112
; GFX9-GISEL-NEXT: global_load_dwordx4 v[28:31], v[0:1], off offset:96
-; GFX9-GISEL-NEXT: s_nop 0
-; GFX9-GISEL-NEXT: global_load_dwordx2 v[0:1], v[0:1], off offset:112
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[4:7], off
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(7)
@@ -5027,10 +5013,9 @@ define void @freeze_v30i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[20:23], off offset:64
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[24:27], off offset:80
-; GFX9-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX9-GISEL-NEXT: s_waitcnt vmcnt(6)
; GFX9-GISEL-NEXT: global_store_dwordx4 v[2:3], v[28:31], off offset:96
-; GFX9-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX9-GISEL-NEXT: global_store_dwordx2 v[2:3], v[0:1], off offset:112
+; GFX9-GISEL-NEXT: global_store_dwordx2 v[2:3], v[32:33], off offset:112
; GFX9-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX9-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -5304,64 +5289,64 @@ define void @freeze_v31i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX803-SDAG-LABEL: freeze_v31i32:
; GFX8-GFX803-SDAG: ; %bb.0:
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x60
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s4, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, 48, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 64, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x70
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v10, vcc, s6, v0
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x50
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x70
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v10, vcc, s4, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v11, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x60
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s5, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x78
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, s6, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx2 v[32:33], v[10:11]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, s5, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s7, 0x78
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v14, vcc, s7, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v15, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: flat_load_dword v34, v[12:13]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v28, vcc, 16, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, 48, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v20, vcc, 32, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dword v34, v[14:15]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s7, 0x50
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s7, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[16:19], v[0:1]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 16, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s4, v2
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s5, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(8)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 64, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s5, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, s7, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s7, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s4, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 32, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v10, vcc, 48, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v11, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[16:19]
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[12:15]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[16:19]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx2 v[4:5], v[32:33]
; GFX8-GFX803-SDAG-NEXT: flat_store_dword v[6:7], v34
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx2 v[0:1], v[32:33]
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(8)
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[8:9], v[20:23]
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(8)
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[10:11], v[24:27]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 16, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[12:15]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(8)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0)
@@ -5372,60 +5357,60 @@ define void @freeze_v31i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 16, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 32, v0
+; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 32, v0
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 48, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 48, v0
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[0:1]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 64, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v35, 0x50
-; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v35
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v37, 0x50
+; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v37
; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x60
-; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v14
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v18, 0x60
+; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v18
; GFX8-GISEL-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x70
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v0, v14
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v18, 0x70
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v18, vcc, v0, v18
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v19, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx3 v[32:34], v[18:19]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
-; GFX8-GISEL-NEXT: flat_load_dwordx3 v[32:34], v[0:1]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 16, v2
+; GFX8-GISEL-NEXT: v_add_u32_e32 v35, vcc, 16, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v36, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[35:36], v[4:7]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 48, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 48, v2
+; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 64, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[8:11]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, v2, v35
-; GFX8-GISEL-NEXT: v_add_u32_e64 v8, s[4:5], 64, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v9, s[4:5], 0, v3, s[4:5]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, v2, v37
; GFX8-GISEL-NEXT: s_mov_b64 s[4:5], vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v10, vcc, 0x60, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v11, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v2, vcc, 0x70, v2
; GFX8-GISEL-NEXT: v_addc_u32_e64 v7, s[4:5], 0, v3, s[4:5]
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 0x60, v2
+; GFX8-GISEL-NEXT: s_mov_b64 s[4:5], vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 0x70, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e64 v1, s[4:5], 0, v3, s[4:5]
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[8:9], v[20:23]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[10:11], v[28:31]
-; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx3 v[2:3], v[32:34]
+; GFX8-GISEL-NEXT: s_waitcnt vmcnt(6)
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
+; GFX8-GISEL-NEXT: flat_store_dwordx3 v[8:9], v[32:34]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX8-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -5752,61 +5737,61 @@ define void @freeze_v32i32(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX803-SDAG-LABEL: freeze_v32i32:
; GFX8-GFX803-SDAG: ; %bb.0:
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x60
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s4, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v32, vcc, 16, v0
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v33, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x60
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s5, v0
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x70
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, s4, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v28, vcc, 48, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x70
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, s5, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, 32, v0
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x50
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v20, vcc, s6, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, 64, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x50
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[16:19], v[0:1]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v16, vcc, s6, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[32:35], v[32:33]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v36, vcc, s5, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v37, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s5, v2
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[36:37], v[4:7]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 64, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, 64, v2
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s6, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, 32, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 48, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 48, v2
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 16, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v10, vcc, 16, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v11, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[12:15]
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[16:19]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[6:7], v[20:23]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[8:9], v[28:31]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[10:11], v[32:35]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[8:9], v[32:35]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0)
; GFX8-GFX803-SDAG-NEXT: s_setpc_b64 s[30:31]
;
@@ -10276,119 +10261,122 @@ define void @freeze_v16p0(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX803-SDAG-LABEL: freeze_v16p0:
; GFX8-GFX803-SDAG: ; %bb.0:
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, 16, v0
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v32, vcc, 16, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v33, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x60
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s5, v0
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x70
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 48, v0
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, s4, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, 32, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x50
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v16, vcc, s4, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x70
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, s5, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x60
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v28, vcc, s6, v0
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v28, vcc, 48, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[32:35], v[0:1]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, 32, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, 64, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x50
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v16, vcc, s6, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[32:35], v[32:33]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v36, vcc, s5, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v37, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s5, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[36:37], v[4:7]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 64, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[20:23]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s4, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[16:19]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 32, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, 32, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 48, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 16, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[32:35]
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 16, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[8:9], v[32:35]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0)
; GFX8-GFX803-SDAG-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-GISEL-LABEL: freeze_v16p0:
; GFX8-GISEL: ; %bb.0:
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 16, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 16, v0
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 32, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 48, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
; GFX8-GISEL-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[0:1]
; GFX8-GISEL-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v38, 0x50
-; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v38
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x60
-; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v14
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x70
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v0, v14
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v36, 0x50
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v36
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v28, 0x60
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v28
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v32, 0x70
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v0, v32
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[32:35], v[0:1]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v36, vcc, 16, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v37, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 16, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[36:37], v[4:7]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[4:7]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[8:11]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 48, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e64 v8, s[4:5], 64, v2
-; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, v2, v38
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v9, s[4:5], 0, v3, s[4:5]
-; GFX8-GISEL-NEXT: s_mov_b64 s[4:5], vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v10, vcc, 0x60, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v11, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v2, vcc, 0x70, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v7, s[4:5], 0, v3, s[4:5]
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 48, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[16:19]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 64, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[8:9], v[20:23]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[20:23]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v2, v36
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 0x60, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[10:11], v[28:31]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 0x70, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[32:35]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[32:35]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX8-GISEL-NEXT: s_setpc_b64 s[30:31]
;
@@ -11501,119 +11489,122 @@ define void @freeze_v16p1(ptr addrspace(1) %ptra, ptr addrspace(1) %ptrb) {
; GFX8-GFX803-SDAG-LABEL: freeze_v16p1:
; GFX8-GFX803-SDAG: ; %bb.0:
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, 16, v0
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v32, vcc, 16, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v33, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x60
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s5, v0
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x70
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 48, v0
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, s4, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, 32, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s4, 0x50
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v16, vcc, s4, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s5, 0x70
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, s5, v0
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x60
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v28, vcc, s6, v0
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v28, vcc, 48, v0
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[32:35], v[0:1]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v24, vcc, 32, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v12, vcc, 64, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
+; GFX8-GFX803-SDAG-NEXT: s_movk_i32 s6, 0x50
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v16, vcc, s6, v0
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
+; GFX8-GFX803-SDAG-NEXT: flat_load_dwordx4 v[32:35], v[32:33]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v36, vcc, s5, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v37, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s5, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[36:37], v[4:7]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 64, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[20:23]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, s4, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[16:19]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 32, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v4, vcc, s6, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v6, vcc, 32, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 48, v2
; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(5)
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
-; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v0, vcc, 16, v2
-; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[32:35]
-; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
+; GFX8-GFX803-SDAG-NEXT: v_add_u32_e32 v8, vcc, 16, v2
+; GFX8-GFX803-SDAG-NEXT: v_addc_u32_e32 v9, vcc, 0, v3, vcc
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[2:3], v[20:23]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
+; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(7)
+; GFX8-GFX803-SDAG-NEXT: flat_store_dwordx4 v[8:9], v[32:35]
; GFX8-GFX803-SDAG-NEXT: s_waitcnt vmcnt(0)
; GFX8-GFX803-SDAG-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-GISEL-LABEL: freeze_v16p1:
; GFX8-GISEL: ; %bb.0:
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 16, v0
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_add_u32_e32 v8, vcc, 16, v0
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v9, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
; GFX8-GISEL-NEXT: v_add_u32_e32 v12, vcc, 32, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v13, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
; GFX8-GISEL-NEXT: v_add_u32_e32 v16, vcc, 48, v0
; GFX8-GISEL-NEXT: v_addc_u32_e32 v17, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
; GFX8-GISEL-NEXT: v_add_u32_e32 v20, vcc, 64, v0
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[8:11], v[0:1]
; GFX8-GISEL-NEXT: v_addc_u32_e32 v21, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v38, 0x50
-; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v38
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x60
-; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v14
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: v_mov_b32_e32 v14, 0x70
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v0, v14
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[12:15], v[12:13]
-; GFX8-GISEL-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v36, 0x50
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[20:23], v[20:21]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v24, vcc, v0, v36
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v25, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v28, 0x60
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[24:27], v[24:25]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v28, vcc, v0, v28
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v29, vcc, 0, v1, vcc
+; GFX8-GISEL-NEXT: v_mov_b32_e32 v32, 0x70
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[28:31], v[28:29]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v0, v32
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-GISEL-NEXT: flat_load_dwordx4 v[32:35], v[0:1]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v36, vcc, 16, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v37, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 16, v2
; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[36:37], v[4:7]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[4:7]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[8:11]
-; GFX8-GISEL-NEXT: v_add_u32_e32 v4, vcc, 48, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e64 v8, s[4:5], 64, v2
-; GFX8-GISEL-NEXT: v_add_u32_e32 v6, vcc, v2, v38
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v9, s[4:5], 0, v3, s[4:5]
-; GFX8-GISEL-NEXT: s_mov_b64 s[4:5], vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v10, vcc, 0x60, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v11, vcc, 0, v3, vcc
-; GFX8-GISEL-NEXT: v_add_u32_e32 v2, vcc, 0x70, v2
-; GFX8-GISEL-NEXT: v_addc_u32_e64 v7, s[4:5], 0, v3, s[4:5]
-; GFX8-GISEL-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 32, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 48, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[4:5], v[16:19]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[16:19]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 64, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[8:9], v[20:23]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[20:23]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, v2, v36
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[6:7], v[24:27]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 0x60, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[10:11], v[28:31]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[28:31]
+; GFX8-GISEL-NEXT: v_add_u32_e32 v0, vcc, 0x70, v2
+; GFX8-GISEL-NEXT: v_addc_u32_e32 v1, vcc, 0, v3, vcc
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(7)
-; GFX8-GISEL-NEXT: flat_store_dwordx4 v[2:3], v[32:35]
+; GFX8-GISEL-NEXT: flat_store_dwordx4 v[0:1], v[32:35]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
; GFX8-GISEL-NEXT: s_setpc_b64 s[30:31]
;
diff --git a/llvm/test/CodeGen/AMDGPU/gfx-callable-argument-types.ll b/llvm/test/CodeGen/AMDGPU/gfx-callable-argument-types.ll
index c6105e2eda9f3..ced30db4f8248 100644
--- a/llvm/test/CodeGen/AMDGPU/gfx-callable-argument-types.ll
+++ b/llvm/test/CodeGen/AMDGPU/gfx-callable-argument-types.ll
@@ -8288,6 +8288,8 @@ define amdgpu_gfx void @test_call_external_void_func_v32i32() #0 {
; GFX9-NEXT: v_writelane_b32 v40, s31, 1
; GFX9-NEXT: s_load_dwordx2 s[34:35], s[34:35], 0x0
; GFX9-NEXT: v_mov_b32_e32 v28, 0
+; GFX9-NEXT: s_mov_b32 s37, external_void_func_v32i32 at abs32@hi
+; GFX9-NEXT: s_mov_b32 s36, external_void_func_v32i32 at abs32@lo
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: global_load_dwordx4 v[0:3], v28, s[34:35]
; GFX9-NEXT: global_load_dwordx4 v[4:7], v28, s[34:35] offset:16
@@ -8298,9 +8300,7 @@ define amdgpu_gfx void @test_call_external_void_func_v32i32() #0 {
; GFX9-NEXT: global_load_dwordx4 v[24:27], v28, s[34:35] offset:96
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: global_load_dwordx4 v[28:31], v28, s[34:35] offset:112
-; GFX9-NEXT: s_mov_b32 s35, external_void_func_v32i32 at abs32@hi
-; GFX9-NEXT: s_mov_b32 s34, external_void_func_v32i32 at abs32@lo
-; GFX9-NEXT: s_swappc_b64 s[30:31], s[34:35]
+; GFX9-NEXT: s_swappc_b64 s[30:31], s[36:37]
; GFX9-NEXT: v_readlane_b32 s30, v40, 0
; GFX9-NEXT: v_readlane_b32 s31, v40, 1
; GFX9-NEXT: s_mov_b32 s32, s33
diff --git a/llvm/test/CodeGen/AMDGPU/high-RP-reschedule.mir b/llvm/test/CodeGen/AMDGPU/high-RP-reschedule.mir
index 604c71f40a4b3..d4087db4810ad 100644
--- a/llvm/test/CodeGen/AMDGPU/high-RP-reschedule.mir
+++ b/llvm/test/CodeGen/AMDGPU/high-RP-reschedule.mir
@@ -10,10 +10,12 @@
# GCN: Unclustered High Register Pressure Reschedule stage successfully increased occupancy to 4
-# GCN-GCNTRACKER: Occupancy before scheduling: 3, after 4.
-# GCN-GCNTRACKER-NEXT: Ending scheduling stage: Max Occupancy Initial Schedule
+# GCN-GCNTRACKER: Starting scheduling stage: Max Occupancy Initial Schedule
+# GCN-GCNTRACKER-NOT: Occupancy before scheduling: 3
+# GCN-GCNTRACKER: Ending scheduling stage: Max Occupancy Initial Schedule
# When using the GCN Trackers, the scheduler is able to acieve desired occupancy without running high-RP-reschedule stage.
+# The initial schedule already brings the region within the critical VGPR limit, so occupancy never drops to 3.
---
name: high-RP-reschedule
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.exp.f64.ll b/llvm/test/CodeGen/AMDGPU/llvm.exp.f64.ll
index 1104362f3e995..225ea13e4c30e 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.exp.f64.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.exp.f64.ll
@@ -1046,125 +1046,125 @@ define <3 x double> @v_exp_v3f64(<3 x double> %in) #0 {
; SI-GISEL-LABEL: v_exp_v3f64:
; SI-GISEL: ; %bb.0:
; SI-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0x652b82fe
-; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x3ff71547
-; SI-GISEL-NEXT: v_mul_f64 v[12:13], v[0:1], v[10:11]
-; SI-GISEL-NEXT: v_mov_b32_e32 v6, 0
-; SI-GISEL-NEXT: v_and_b32_e32 v7, 0x80000000, v13
-; SI-GISEL-NEXT: v_or_b32_e32 v7, 0x43300000, v7
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[12:13], v[6:7]
-; SI-GISEL-NEXT: v_mov_b32_e32 v8, -1
-; SI-GISEL-NEXT: v_mov_b32_e32 v9, 0x432fffff
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[14:15], -v[6:7]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[12:13]|, v[8:9]
+; SI-GISEL-NEXT: v_mov_b32_e32 v6, 0x652b82fe
+; SI-GISEL-NEXT: v_mov_b32_e32 v7, 0x3ff71547
+; SI-GISEL-NEXT: v_mul_f64 v[8:9], v[0:1], v[6:7]
+; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0
+; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v9
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
+; SI-GISEL-NEXT: v_add_f64 v[12:13], v[8:9], v[10:11]
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, -1
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x432fffff
+; SI-GISEL-NEXT: v_add_f64 v[11:12], v[12:13], -v[10:11]
+; SI-GISEL-NEXT: v_mul_f64 v[16:17], v[2:3], v[6:7]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[8:9]|, v[14:15]
+; SI-GISEL-NEXT: v_mul_f64 v[6:7], v[4:5], v[6:7]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v11, v8, vcc
+; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v17
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
+; SI-GISEL-NEXT: v_add_f64 v[18:19], v[16:17], v[10:11]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v12, v9, vcc
+; SI-GISEL-NEXT: v_add_f64 v[11:12], v[18:19], -v[10:11]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[16:17]|, v[14:15]
; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0xfefa39ef
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v12, v14, v12, vcc
; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x3fe62e42
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v13, v15, v13, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], -v[12:13], v[18:19], v[0:1]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v11, v16, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v12, v17, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[11:12], -v[8:9], v[18:19], v[0:1]
; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x3b39803f
; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x3c7abc9e
-; SI-GISEL-NEXT: v_mul_f64 v[16:17], v[2:3], v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], -v[12:13], v[20:21], v[14:15]
-; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0x6a5dcb37
-; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3e5ade15
-; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0xfca7ab0c
-; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0x3e928af3
-; SI-GISEL-NEXT: v_and_b32_e32 v7, 0x80000000, v17
-; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[14:15], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_or_b32_e32 v7, 0x43300000, v7
-; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x623fde64
-; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3ec71dee
-; SI-GISEL-NEXT: v_add_f64 v[22:23], v[16:17], v[6:7]
-; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[14:15], v[28:29], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], -v[8:9], v[20:21], v[11:12]
+; SI-GISEL-NEXT: v_and_b32_e32 v26, 0x80000000, v7
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0x6a5dcb37
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x3e5ade15
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0xfca7ab0c
+; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3e928af3
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v26
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[12:13], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x623fde64
+; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3ec71dee
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[12:13], v[26:27], v[28:29]
+; SI-GISEL-NEXT: v_add_f64 v[30:31], v[6:7], v[10:11]
; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0x7c89e6b0
; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3efa0199
-; SI-GISEL-NEXT: v_add_f64 v[22:23], v[22:23], -v[6:7]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[16:17]|, v[8:9]
-; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[14:15], v[28:29], v[32:33]
-; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x14761f6e
-; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3f2a01a0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v22, v16, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v23, v17, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[28:29], v[34:35]
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x1852b7b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3f56c16c
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[28:29]
-; SI-GISEL-NEXT: v_mul_f64 v[10:11], v[4:5], v[10:11]
-; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x11122322
-; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3f811111
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[36:37]
-; SI-GISEL-NEXT: v_and_b32_e32 v7, 0x80000000, v11
-; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x555502a1
-; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3fa55555
-; SI-GISEL-NEXT: v_or_b32_e32 v7, 0x43300000, v7
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[38:39]
-; SI-GISEL-NEXT: v_add_f64 v[48:49], v[10:11], v[6:7]
-; SI-GISEL-NEXT: v_mov_b32_e32 v50, 0x55555511
-; SI-GISEL-NEXT: v_mov_b32_e32 v51, 0x3fc55555
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[50:51]
-; SI-GISEL-NEXT: v_add_f64 v[6:7], v[48:49], -v[6:7]
-; SI-GISEL-NEXT: v_mov_b32_e32 v48, 11
-; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3fe00000
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[48:49]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[10:11]|, v[8:9]
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[14:15], v[22:23], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[16:17], v[18:19], v[2:3]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, v6, v10, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v7, v11, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[12:13], v[26:27], v[32:33]
+; SI-GISEL-NEXT: v_add_f64 v[10:11], v[30:31], -v[10:11]
+; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x14761f6e
+; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3f2a01a0
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[6:7]|, v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[12:13], v[26:27], v[30:31]
+; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0x1852b7b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0x3f56c16c
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, v10, v6, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v11, v7, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[14:15], v[26:27]
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0x11122322
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x3f811111
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x555502a1
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3fa55555
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[34:35]
+; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x55555511
+; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3fc55555
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[36:37]
+; SI-GISEL-NEXT: v_mov_b32_e32 v38, 11
+; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3fe00000
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[38:39]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], -v[16:17], v[18:19], v[2:3]
; SI-GISEL-NEXT: v_fma_f64 v[18:19], -v[6:7], v[18:19], v[4:5]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[16:17], v[20:21], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], -v[16:17], v[20:21], v[12:13]
; SI-GISEL-NEXT: v_fma_f64 v[18:19], -v[6:7], v[20:21], v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[18:19], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[18:19], v[24:25], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[18:19], v[24:25], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[34:35]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v26, v[12:13]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[18:19], v[24:25], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[20:21], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[18:19], v[24:25], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[14:15], v[8:9], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[24:25], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[48:49]
-; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x40900000
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v26
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v16, v[16:17]
-; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0xc090cc00
-; SI-GISEL-NEXT: v_mov_b32_e32 v17, 0x7ff00000
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[12:13], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[10:11], v8
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[20:21], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[18:19], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[22:23], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[26:27]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[26:27]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[18:19], v[20:21], v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[18:19], v[14:15], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[18:19], v[14:15], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], 1.0
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x40900000
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[18:19], v[14:15], v[38:39]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v14, v[16:17]
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0xc090cc00
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x7ff00000
; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, 0, v8, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v22, v17, v9, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[48:49]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v15, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[24:25]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v8, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[12:13], v16
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v22, vcc
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[2:3], v[10:11]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v16, v[6:7]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v12, 0, v8, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v13, v17, v9, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[18:19], v[20:21], 1.0
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[14:15]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v9, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[10:11], v14
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[2:3], v[22:23]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, 0, v8, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v15, v9, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[18:19], v[12:13], 1.0
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v12, v[6:7]
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[24:25]
; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[18:19], v[8:9], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v12, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v16
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v13, vcc
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[10:11]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v10, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v11, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v12
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[22:23]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v6, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v17, v7, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[14:15]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v15, v7, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[24:25]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v6, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v7, vcc
; SI-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -1755,154 +1755,154 @@ define <4 x double> @v_exp_v4f64(<4 x double> %in) #0 {
; SI-GISEL-LABEL: v_exp_v4f64:
; SI-GISEL: ; %bb.0:
; SI-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0x652b82fe
-; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x3ff71547
-; SI-GISEL-NEXT: v_mul_f64 v[14:15], v[0:1], v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v8, 0
-; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v15
-; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
-; SI-GISEL-NEXT: v_add_f64 v[16:17], v[14:15], v[8:9]
-; SI-GISEL-NEXT: v_mul_f64 v[18:19], v[2:3], v[12:13]
-; SI-GISEL-NEXT: v_add_f64 v[16:17], v[16:17], -v[8:9]
-; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v19
-; SI-GISEL-NEXT: v_mov_b32_e32 v10, -1
-; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x432fffff
-; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[14:15]|, v[10:11]
-; SI-GISEL-NEXT: v_add_f64 v[20:21], v[18:19], v[8:9]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v16, v14, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v17, v15, vcc
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[20:21], -v[8:9]
-; SI-GISEL-NEXT: v_mul_f64 v[20:21], v[4:5], v[12:13]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[18:19]|, v[10:11]
-; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v21
-; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
-; SI-GISEL-NEXT: v_add_f64 v[22:23], v[20:21], v[8:9]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v14, v14, v18, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v15, v15, v19, vcc
-; SI-GISEL-NEXT: v_add_f64 v[18:19], v[22:23], -v[8:9]
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0x652b82fe
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x3ff71547
+; SI-GISEL-NEXT: v_mul_f64 v[8:9], v[0:1], v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0
+; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v9
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
+; SI-GISEL-NEXT: v_add_f64 v[16:17], v[8:9], v[10:11]
+; SI-GISEL-NEXT: v_mul_f64 v[18:19], v[2:3], v[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[16:17], v[16:17], -v[10:11]
+; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v19
+; SI-GISEL-NEXT: v_mov_b32_e32 v12, -1
+; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x432fffff
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[8:9]|, v[12:13]
+; SI-GISEL-NEXT: v_add_f64 v[20:21], v[18:19], v[10:11]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v17, v9, vcc
+; SI-GISEL-NEXT: v_add_f64 v[16:17], v[20:21], -v[10:11]
+; SI-GISEL-NEXT: v_mul_f64 v[20:21], v[4:5], v[14:15]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[18:19]|, v[12:13]
+; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v21
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
+; SI-GISEL-NEXT: v_add_f64 v[22:23], v[20:21], v[10:11]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v16, v18, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v17, v19, vcc
+; SI-GISEL-NEXT: v_add_f64 v[18:19], v[22:23], -v[10:11]
; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0xfefa39ef
; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x3fe62e42
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[20:21]|, v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], -v[16:17], v[22:23], v[0:1]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], -v[8:9], v[22:23], v[0:1]
; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0x3b39803f
; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0x3c7abc9e
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], -v[8:9], v[26:27], v[24:25]
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x6a5dcb37
+; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3e5ade15
+; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0xfca7ab0c
+; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3e928af3
+; SI-GISEL-NEXT: v_fma_f64 v[32:33], v[24:25], v[28:29], v[30:31]
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x623fde64
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3ec71dee
+; SI-GISEL-NEXT: v_fma_f64 v[32:33], v[24:25], v[32:33], v[34:35]
+; SI-GISEL-NEXT: v_mul_f64 v[14:15], v[6:7], v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x7c89e6b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3efa0199
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[20:21]|, v[12:13]
+; SI-GISEL-NEXT: v_fma_f64 v[32:33], v[24:25], v[32:33], v[36:37]
+; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v15
+; SI-GISEL-NEXT: v_mov_b32_e32 v48, 0x14761f6e
+; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3f2a01a0
; SI-GISEL-NEXT: v_cndmask_b32_e32 v18, v18, v20, vcc
+; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
; SI-GISEL-NEXT: v_cndmask_b32_e32 v19, v19, v21, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], -v[16:17], v[26:27], v[24:25]
-; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0x6a5dcb37
-; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3e5ade15
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0xfca7ab0c
-; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3e928af3
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[24:25], v[28:29]
-; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0x623fde64
-; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3ec71dee
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_mul_f64 v[12:13], v[6:7], v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x7c89e6b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3efa0199
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[34:35]
-; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v13
-; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x14761f6e
-; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3f2a01a0
-; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[36:37]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[12:13], v[8:9]
-; SI-GISEL-NEXT: v_mov_b32_e32 v48, 0x1852b7b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3f56c16c
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[48:49]
-; SI-GISEL-NEXT: v_add_f64 v[8:9], v[38:39], -v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[24:25], v[32:33], v[48:49]
+; SI-GISEL-NEXT: v_add_f64 v[38:39], v[14:15], v[10:11]
+; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0x1852b7b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3f56c16c
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[24:25], v[20:21], v[32:33]
+; SI-GISEL-NEXT: v_add_f64 v[10:11], v[38:39], -v[10:11]
; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x11122322
; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3f811111
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[12:13]|, v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[30:31], v[38:39]
-; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x555502a1
-; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3fa55555
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], v[30:31]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v8, v12, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v9, v13, vcc
-; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0x55555511
-; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x3fc55555
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], v[12:13]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[14:15]|, v[12:13]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[24:25], v[20:21], v[38:39]
+; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x555502a1
+; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x3fa55555
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[24:25], v[12:13], v[20:21]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, v10, v14, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v11, v15, vcc
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0x55555511
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x3fc55555
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[24:25], v[12:13], v[14:15]
; SI-GISEL-NEXT: v_mov_b32_e32 v50, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v51, 0x3fe00000
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], v[50:51]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v52, v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v20, v[16:17]
-; SI-GISEL-NEXT: v_fma_f64 v[16:17], -v[14:15], v[22:23], v[2:3]
-; SI-GISEL-NEXT: v_fma_f64 v[16:17], -v[14:15], v[26:27], v[16:17]
-; SI-GISEL-NEXT: v_ldexp_f64 v[10:11], v[10:11], v20
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[24:25], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[12:13]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[16:17], v[20:21], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], -v[18:19], v[22:23], v[4:5]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[8:9], v[22:23], v[6:7]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], -v[18:19], v[26:27], v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[8:9], v[26:27], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[24:25], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[26:27], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[48:49]
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[38:39]
-; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x40900000
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[30:31]
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[12:13]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], v[50:51]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v53, 0, v10, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v10, v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[24:25], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[24:25], v[12:13], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[24:25], v[12:13], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[24:25], v[12:13], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], -v[16:17], v[22:23], v[2:3]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], -v[16:17], v[26:27], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[28:29], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], v[24:25], v[52:53], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[24:25], v[52:53], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], -v[18:19], v[22:23], v[4:5]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[10:11], v[22:23], v[6:7]
+; SI-GISEL-NEXT: v_fma_f64 v[52:53], -v[18:19], v[26:27], v[52:53]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[10:11], v[26:27], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[52:53], v[28:29], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[28:29], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[52:53], v[26:27], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[28:29], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[52:53], v[26:27], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[28:29], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[52:53], v[26:27], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[28:29], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[52:53], v[26:27], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[28:29], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[52:53], v[26:27], v[38:39]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v32, v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[52:53], v[26:27], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[28:29], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[52:53], v[8:9], v[14:15]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v33, v[16:17]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[22:23], v[28:29], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[52:53], v[8:9], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[22:23], v[16:17], v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[52:53], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v17, v[18:19]
+; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x40900000
+; SI-GISEL-NEXT: v_ldexp_f64 v[12:13], v[12:13], v32
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[52:53], v[8:9], 1.0
; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0x7ff00000
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[24:25], v[12:13]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v19, v18, v11, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[10:11], v[14:15], v10
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[28:29]
-; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0xc090cc00
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[50:51]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, 0, v10, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v18, v11, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[26:27]
-; SI-GISEL-NEXT: v_ldexp_f64 v[16:17], v[16:17], v52
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e64 s[4:5], v[2:3], v[28:29]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v14, v[8:9]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v53, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v19, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[22:23], v[12:13], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e64 v16, 0, v16, s[4:5]
-; SI-GISEL-NEXT: v_cndmask_b32_e64 v17, v18, v17, s[4:5]
+; SI-GISEL-NEXT: v_ldexp_f64 v[20:21], v[24:25], v33
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v12, 0, v12, vcc
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e64 s[4:5], v[2:3], v[30:31]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v13, v18, v13, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v17
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[30:31]
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0xc090cc00
+; SI-GISEL-NEXT: v_cndmask_b32_e64 v16, 0, v20, s[4:5]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[22:23], v[14:15], v[50:51]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v19, 0, v8, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v20, v18, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[34:35]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v10, v[10:11]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v12, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v13, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[22:23], v[14:15], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e64 v17, v18, v21, s[4:5]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v16, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v17, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[26:27]
; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[22:23], v[8:9], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v10, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v11, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v14
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[6:7], v[28:29]
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[34:35]
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v10
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v19, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v20, vcc
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[6:7], v[30:31]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, 0, v8, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v18, v9, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[6:7], v[26:27]
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[6:7], v[34:35]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v8, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, 0, v9, vcc
; SI-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -3615,12 +3615,15 @@ define amdgpu_ps <6 x i32> @s_exp_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_mov_b32_e32 v30, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3fe00000
; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[4:5], v[14:15], v[30:31]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v32, s[10:11]
+; SI-GISEL-NEXT: v_mul_f64 v[32:33], s[2:3], v[0:1]
; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[4:5], v[14:15], 1.0
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[32:33]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[4:5], v[4:5], v[14:15], 1.0
-; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0
-; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v32
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v14, s[10:11]
; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x40900000
+; SI-GISEL-NEXT: v_mul_f64 v[0:1], s[4:5], v[0:1]
+; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v14
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[0:1], v[14:15]
; SI-GISEL-NEXT: v_readfirstlane_b32 s12, v4
; SI-GISEL-NEXT: v_readfirstlane_b32 s13, v5
@@ -3630,7 +3633,6 @@ define amdgpu_ps <6 x i32> @s_exp_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[0:1], v[4:5]
; SI-GISEL-NEXT: s_mov_b32 s10, 0
; SI-GISEL-NEXT: s_mov_b32 s11, 0x7ff00000
-; SI-GISEL-NEXT: v_mul_f64 v[32:33], s[2:3], v[0:1]
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[12:13], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[12:13], vcc, vcc
; SI-GISEL-NEXT: v_readfirstlane_b32 s12, v32
@@ -3638,22 +3640,21 @@ define amdgpu_ps <6 x i32> @s_exp_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0
; SI-GISEL-NEXT: s_and_b64 s[14:15], s[12:13], s[8:9]
; SI-GISEL-NEXT: s_or_b64 s[14:15], s[14:15], s[6:7]
-; SI-GISEL-NEXT: v_add_f64 v[34:35], v[32:33], s[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], s[14:15]
+; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], -s[14:15]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[2:3], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[34:35], v[34:35], -s[14:15]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[14:15], |v[32:33]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s16, v34
-; SI-GISEL-NEXT: v_readfirstlane_b32 s17, v35
-; SI-GISEL-NEXT: s_or_b64 s[14:15], s[14:15], s[14:15]
-; SI-GISEL-NEXT: s_cselect_b64 s[12:13], s[12:13], s[16:17]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v32
+; SI-GISEL-NEXT: v_readfirstlane_b32 s15, v33
+; SI-GISEL-NEXT: s_cselect_b64 s[12:13], s[12:13], s[14:15]
; SI-GISEL-NEXT: v_mov_b32_e32 v33, s13
; SI-GISEL-NEXT: v_mov_b32_e32 v32, s12
; SI-GISEL-NEXT: v_fma_f64 v[32:33], -v[32:33], v[2:3], s[2:3]
-; SI-GISEL-NEXT: s_or_b64 s[14:15], vcc, vcc
+; SI-GISEL-NEXT: s_or_b64 s[16:17], vcc, vcc
; SI-GISEL-NEXT: v_fma_f64 v[32:33], -s[12:13], v[8:9], v[32:33]
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[2:3], v[4:5]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[10:11], v[12:13]
-; SI-GISEL-NEXT: v_mul_f64 v[0:1], s[4:5], v[0:1]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[14:15], |v[0:1]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[16:17]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[18:19]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[20:21]
@@ -3675,14 +3676,13 @@ define amdgpu_ps <6 x i32> @s_exp_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[2:3], s[2:3], 0
; SI-GISEL-NEXT: s_and_b64 s[8:9], s[12:13], s[8:9]
; SI-GISEL-NEXT: s_or_b64 s[6:7], s[8:9], s[6:7]
-; SI-GISEL-NEXT: v_add_f64 v[32:33], v[0:1], s[6:7]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], s[6:7]
+; SI-GISEL-NEXT: s_or_b64 s[8:9], s[14:15], s[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], -s[6:7]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[4:5], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], -s[6:7]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[6:7], |v[0:1]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s8, v32
-; SI-GISEL-NEXT: v_readfirstlane_b32 s9, v33
-; SI-GISEL-NEXT: s_or_b64 s[6:7], s[6:7], s[6:7]
-; SI-GISEL-NEXT: s_cselect_b64 s[6:7], s[12:13], s[8:9]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s6, v0
+; SI-GISEL-NEXT: v_readfirstlane_b32 s7, v1
+; SI-GISEL-NEXT: s_cselect_b64 s[6:7], s[12:13], s[6:7]
; SI-GISEL-NEXT: v_mov_b32_e32 v0, s6
; SI-GISEL-NEXT: v_mov_b32_e32 v1, s7
; SI-GISEL-NEXT: v_fma_f64 v[0:1], -v[0:1], v[2:3], s[4:5]
@@ -4407,12 +4407,14 @@ define amdgpu_ps <8 x i32> @s_exp_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_mov_b32_e32 v30, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3fe00000
; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[4:5], v[14:15], v[30:31]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v32, s[12:13]
+; SI-GISEL-NEXT: v_mul_f64 v[32:33], s[2:3], v[0:1]
; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[4:5], v[14:15], 1.0
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[18:19], |v[32:33]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[4:5], v[4:5], v[14:15], 1.0
-; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0
-; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v32
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v14, s[12:13]
; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x40900000
+; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v14
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[0:1], v[14:15]
; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v4
; SI-GISEL-NEXT: v_readfirstlane_b32 s15, v5
@@ -4422,7 +4424,6 @@ define amdgpu_ps <8 x i32> @s_exp_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[0:1], v[4:5]
; SI-GISEL-NEXT: s_mov_b32 s12, 0
; SI-GISEL-NEXT: s_mov_b32 s13, 0x7ff00000
-; SI-GISEL-NEXT: v_mul_f64 v[32:33], s[2:3], v[0:1]
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[14:15], s[12:13]
; SI-GISEL-NEXT: s_or_b64 s[14:15], vcc, vcc
; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v32
@@ -4430,14 +4431,13 @@ define amdgpu_ps <8 x i32> @s_exp_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0
; SI-GISEL-NEXT: s_and_b64 s[16:17], s[14:15], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[34:35], v[32:33], s[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], s[16:17]
+; SI-GISEL-NEXT: s_or_b64 s[18:19], s[18:19], s[18:19]
+; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], -s[16:17]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[2:3], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[34:35], v[34:35], -s[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[32:33]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s18, v34
-; SI-GISEL-NEXT: v_readfirstlane_b32 s19, v35
-; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[16:17]
-; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[18:19]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s16, v32
+; SI-GISEL-NEXT: v_readfirstlane_b32 s17, v33
+; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[16:17]
; SI-GISEL-NEXT: v_mov_b32_e32 v33, s15
; SI-GISEL-NEXT: v_mov_b32_e32 v32, s14
; SI-GISEL-NEXT: v_fma_f64 v[32:33], -v[32:33], v[2:3], s[2:3]
@@ -4467,23 +4467,24 @@ define amdgpu_ps <8 x i32> @s_exp_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[2:3], s[2:3], 0
; SI-GISEL-NEXT: s_and_b64 s[16:17], s[14:15], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[34:35], v[32:33], s[16:17]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[18:19], |v[32:33]|, v[6:7]
+; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], s[16:17]
+; SI-GISEL-NEXT: s_or_b64 s[18:19], s[18:19], s[18:19]
+; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], -s[16:17]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[4:5], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[34:35], v[34:35], -s[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[32:33]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s18, v34
-; SI-GISEL-NEXT: v_readfirstlane_b32 s19, v35
-; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[16:17]
-; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[18:19]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s16, v32
+; SI-GISEL-NEXT: v_readfirstlane_b32 s17, v33
+; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[16:17]
; SI-GISEL-NEXT: v_mov_b32_e32 v33, s15
; SI-GISEL-NEXT: v_mov_b32_e32 v32, s14
; SI-GISEL-NEXT: v_fma_f64 v[32:33], -v[32:33], v[2:3], s[4:5]
-; SI-GISEL-NEXT: s_or_b64 s[16:17], vcc, vcc
+; SI-GISEL-NEXT: s_or_b64 s[18:19], vcc, vcc
; SI-GISEL-NEXT: v_fma_f64 v[32:33], -s[14:15], v[8:9], v[32:33]
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[4:5], v[4:5]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[10:11], v[12:13]
; SI-GISEL-NEXT: v_mul_f64 v[0:1], s[6:7], v[0:1]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[16:17]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[0:1]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[18:19]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[20:21]
; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[32:33], v[34:35], v[22:23]
@@ -4504,14 +4505,13 @@ define amdgpu_ps <8 x i32> @s_exp_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[4:5], s[4:5], 0
; SI-GISEL-NEXT: s_and_b64 s[10:11], s[14:15], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[8:9], s[10:11], s[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[32:33], v[0:1], s[8:9]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], s[8:9]
+; SI-GISEL-NEXT: s_or_b64 s[10:11], s[16:17], s[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], -s[8:9]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[6:7], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[32:33], v[32:33], -s[8:9]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[8:9], |v[0:1]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s10, v32
-; SI-GISEL-NEXT: v_readfirstlane_b32 s11, v33
-; SI-GISEL-NEXT: s_or_b64 s[8:9], s[8:9], s[8:9]
-; SI-GISEL-NEXT: s_cselect_b64 s[8:9], s[14:15], s[10:11]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s8, v0
+; SI-GISEL-NEXT: v_readfirstlane_b32 s9, v1
+; SI-GISEL-NEXT: s_cselect_b64 s[8:9], s[14:15], s[8:9]
; SI-GISEL-NEXT: v_mov_b32_e32 v0, s8
; SI-GISEL-NEXT: v_mov_b32_e32 v1, s9
; SI-GISEL-NEXT: v_fma_f64 v[0:1], -v[0:1], v[2:3], s[6:7]
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.exp10.f64.ll b/llvm/test/CodeGen/AMDGPU/llvm.exp10.f64.ll
index 22d3ed88015a9..e92527c85f4e7 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.exp10.f64.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.exp10.f64.ll
@@ -1140,135 +1140,135 @@ define <3 x double> @v_exp10_v3f64(<3 x double> %in) #0 {
; SI-GISEL-LABEL: v_exp10_v3f64:
; SI-GISEL: ; %bb.0:
; SI-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0x979a371
-; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x400a934f
-; SI-GISEL-NEXT: v_mul_f64 v[6:7], v[0:1], v[10:11]
-; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0
-; SI-GISEL-NEXT: v_and_b32_e32 v8, 0x80000000, v7
-; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v8
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[6:7], v[12:13]
-; SI-GISEL-NEXT: v_mul_f64 v[16:17], v[2:3], v[10:11]
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[14:15], -v[12:13]
-; SI-GISEL-NEXT: v_and_b32_e32 v13, 0x80000000, v17
-; SI-GISEL-NEXT: v_mov_b32_e32 v8, -1
-; SI-GISEL-NEXT: v_mov_b32_e32 v9, 0x432fffff
-; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v13
-; SI-GISEL-NEXT: v_add_f64 v[18:19], v[16:17], v[12:13]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[6:7]|, v[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[18:19], v[18:19], -v[12:13]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v14, v14, v6, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v15, v15, v7, vcc
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[16:17]|, v[8:9]
-; SI-GISEL-NEXT: v_mul_f64 v[10:11], v[4:5], v[10:11]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, v18, v16, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v19, v17, vcc
-; SI-GISEL-NEXT: v_mov_b32_e32 v16, 0x509f79ff
-; SI-GISEL-NEXT: v_mov_b32_e32 v17, 0x3fd34413
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], -v[14:15], v[16:17], v[0:1]
-; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0xa994fd21
-; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0xbc49dc1d
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], -v[14:15], v[20:21], v[18:19]
-; SI-GISEL-NEXT: v_and_b32_e32 v13, 0x80000000, v11
-; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0x494ea3e9
-; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0xbcaf48ad
-; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v13
-; SI-GISEL-NEXT: v_mul_f64 v[26:27], v[18:19], v[22:23]
-; SI-GISEL-NEXT: v_add_f64 v[24:25], v[10:11], v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0xbbb55516
-; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x40026bb1
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[18:19], v[28:29], v[26:27]
-; SI-GISEL-NEXT: v_add_f64 v[12:13], v[24:25], -v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0x6a5dcb37
-; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3e5ade15
-; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0xfca7ab0c
-; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0x3e928af3
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[10:11]|, v[8:9]
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[18:19], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0x623fde64
-; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3ec71dee
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v12, v10, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v13, v11, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], -v[6:7], v[16:17], v[2:3]
-; SI-GISEL-NEXT: v_fma_f64 v[16:17], -v[8:9], v[16:17], v[4:5]
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], -v[6:7], v[20:21], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[16:17], -v[8:9], v[20:21], v[16:17]
-; SI-GISEL-NEXT: v_mul_f64 v[48:49], v[30:31], v[22:23]
-; SI-GISEL-NEXT: v_mul_f64 v[22:23], v[16:17], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[30:31], v[28:29], v[48:49]
-; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0x7c89e6b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x3efa0199
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[12:13]
-; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[16:17], v[28:29], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[30:31], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x14761f6e
-; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3f2a01a0
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[16:17], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[30:31], v[28:29], v[32:33]
-; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x1852b7b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3f56c16c
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[16:17], v[24:25], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[30:31], v[26:27], v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x11122322
-; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3f811111
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[24:25], v[12:13]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[30:31], v[26:27], v[34:35]
-; SI-GISEL-NEXT: v_mov_b32_e32 v50, 0x555502a1
-; SI-GISEL-NEXT: v_mov_b32_e32 v51, 0x3fa55555
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[30:31], v[24:25], v[36:37]
-; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x55555511
-; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x3fc55555
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[20:21]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v26, v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[24:25], v[38:39]
+; SI-GISEL-NEXT: v_mov_b32_e32 v7, 0x979a371
+; SI-GISEL-NEXT: v_mov_b32_e32 v8, 0x400a934f
+; SI-GISEL-NEXT: v_mul_f64 v[9:10], v[0:1], v[7:8]
+; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0
+; SI-GISEL-NEXT: v_and_b32_e32 v6, 0x80000000, v10
+; SI-GISEL-NEXT: v_or_b32_e32 v12, 0x43300000, v6
+; SI-GISEL-NEXT: v_add_f64 v[13:14], v[9:10], v[11:12]
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, -1
+; SI-GISEL-NEXT: v_mov_b32_e32 v16, 0x432fffff
+; SI-GISEL-NEXT: v_mul_f64 v[17:18], v[2:3], v[7:8]
+; SI-GISEL-NEXT: v_add_f64 v[12:13], v[13:14], -v[11:12]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[9:10]|, v[15:16]
+; SI-GISEL-NEXT: v_and_b32_e32 v6, 0x80000000, v18
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v12, v9, vcc
+; SI-GISEL-NEXT: v_or_b32_e32 v12, 0x43300000, v6
+; SI-GISEL-NEXT: v_add_f64 v[19:20], v[17:18], v[11:12]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, v13, v10, vcc
+; SI-GISEL-NEXT: v_add_f64 v[12:13], v[19:20], -v[11:12]
+; SI-GISEL-NEXT: v_mul_f64 v[19:20], v[4:5], v[7:8]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[17:18]|, v[15:16]
+; SI-GISEL-NEXT: v_and_b32_e32 v8, 0x80000000, v20
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, v12, v17, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v13, v18, vcc
+; SI-GISEL-NEXT: v_or_b32_e32 v12, 0x43300000, v8
+; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x509f79ff
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0x3fd34413
+; SI-GISEL-NEXT: v_add_f64 v[17:18], v[19:20], v[11:12]
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], -v[9:10], v[13:14], v[0:1]
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0xa994fd21
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0xbc49dc1d
+; SI-GISEL-NEXT: v_add_f64 v[11:12], v[17:18], -v[11:12]
+; SI-GISEL-NEXT: v_fma_f64 v[17:18], -v[9:10], v[23:24], v[21:22]
+; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x494ea3e9
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0xbcaf48ad
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[19:20]|, v[15:16]
+; SI-GISEL-NEXT: v_mul_f64 v[15:16], v[17:18], v[21:22]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v11, v19, vcc
+; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0xbbb55516
+; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0x40026bb1
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v12, v12, v20, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[25:26], v[15:16]
+; SI-GISEL-NEXT: v_fma_f64 v[17:18], -v[6:7], v[13:14], v[2:3]
+; SI-GISEL-NEXT: v_fma_f64 v[13:14], -v[11:12], v[13:14], v[4:5]
+; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x6a5dcb37
+; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x3e5ade15
+; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0xfca7ab0c
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x3e928af3
+; SI-GISEL-NEXT: v_fma_f64 v[17:18], -v[6:7], v[23:24], v[17:18]
+; SI-GISEL-NEXT: v_fma_f64 v[13:14], -v[11:12], v[23:24], v[13:14]
+; SI-GISEL-NEXT: v_fma_f64 v[23:24], v[15:16], v[19:20], v[27:28]
+; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x623fde64
+; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x3ec71dee
+; SI-GISEL-NEXT: v_fma_f64 v[23:24], v[15:16], v[23:24], v[29:30]
+; SI-GISEL-NEXT: v_mul_f64 v[31:32], v[17:18], v[21:22]
+; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x7c89e6b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x3efa0199
+; SI-GISEL-NEXT: v_fma_f64 v[23:24], v[15:16], v[23:24], v[33:34]
+; SI-GISEL-NEXT: v_fma_f64 v[17:18], v[17:18], v[25:26], v[31:32]
+; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x14761f6e
+; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0x3f2a01a0
+; SI-GISEL-NEXT: v_mul_f64 v[21:22], v[13:14], v[21:22]
+; SI-GISEL-NEXT: v_fma_f64 v[23:24], v[15:16], v[23:24], v[31:32]
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x1852b7b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x3f56c16c
+; SI-GISEL-NEXT: v_fma_f64 v[13:14], v[13:14], v[25:26], v[21:22]
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[15:16], v[23:24], v[35:36]
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x11122322
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0x3f811111
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], v[23:24]
+; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x555502a1
+; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0x3fa55555
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], v[25:26]
+; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x55555511
+; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x3fc55555
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], v[37:38]
; SI-GISEL-NEXT: v_mov_b32_e32 v48, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3fe00000
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[12:13], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[14:15], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[12:13], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[14:15], v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[12:13], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[18:19], v[10:11], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[14:15], v[48:49]
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x40900000
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[12:13], v[50:51]
-; SI-GISEL-NEXT: v_ldexp_f64 v[10:11], v[10:11], v26
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[14:15], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], v[48:49]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[9:10]
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], 1.0
; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v6, v[6:7]
-; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0xc090cc00
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[12:13], v[20:21]
-; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x7ff00000
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, 0, v10, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[14:15], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v20, v11, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[12:13], v[48:49]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v10, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[14:15], v6
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v11, vcc
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[2:3], v[18:19]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, 0, v6, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v20, v7, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[16:17], v[12:13], 1.0
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[16:17], v[6:7], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v10, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v8
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v11, vcc
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[18:19]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[15:16], v[21:22], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[21:22], v[17:18], v[19:20], v[27:28]
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[15:16], v8
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[21:22], v[29:30]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[27:28]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[33:34]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[29:30]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[31:32]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[33:34]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[35:36]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[31:32]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[23:24]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[35:36]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[25:26]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[23:24]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[37:38]
+; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], v[48:49]
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0x40900000
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[21:22]
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[25:26]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], 1.0
+; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0xc090cc00
+; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0x7ff00000
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, 0, v8, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[19:20], v[13:14], v[19:20], v[37:38]
+; SI-GISEL-NEXT: v_fma_f64 v[15:16], v[17:18], v[15:16], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v10, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[27:28]
+; SI-GISEL-NEXT: v_fma_f64 v[17:18], v[13:14], v[19:20], v[48:49]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v8, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v7, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[15:16], v6
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[2:3], v[21:22]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v11, v[11:12]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, 0, v6, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v10, v7, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[13:14], v[17:18], 1.0
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[27:28]
+; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[13:14], v[6:7], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v8, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v9, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v11
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[21:22]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v6, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v20, v7, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[22:23]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v10, v7, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[27:28]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v6, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v7, vcc
; SI-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -1747,164 +1747,164 @@ define <4 x double> @v_exp10_v4f64(<4 x double> %in) #0 {
; SI-SDAG-NEXT: s_mov_b32 s4, 0x979a371
; SI-SDAG-NEXT: s_mov_b32 s5, 0x400a934f
; SI-SDAG-NEXT: v_mul_f64 v[8:9], v[0:1], s[4:5]
-; SI-SDAG-NEXT: s_brev_b32 s8, -2
+; SI-SDAG-NEXT: s_brev_b32 s10, -2
; SI-SDAG-NEXT: v_mov_b32_e32 v16, 0x43300000
-; SI-SDAG-NEXT: v_bfi_b32 v15, s8, v16, v9
-; SI-SDAG-NEXT: v_mov_b32_e32 v14, 0
-; SI-SDAG-NEXT: v_add_f64 v[10:11], v[8:9], v[14:15]
+; SI-SDAG-NEXT: v_bfi_b32 v11, s10, v16, v9
+; SI-SDAG-NEXT: v_mov_b32_e32 v10, 0
+; SI-SDAG-NEXT: v_add_f64 v[12:13], v[8:9], v[10:11]
; SI-SDAG-NEXT: s_mov_b32 s6, -1
; SI-SDAG-NEXT: s_mov_b32 s7, 0x432fffff
-; SI-SDAG-NEXT: v_add_f64 v[10:11], v[10:11], -v[14:15]
+; SI-SDAG-NEXT: v_add_f64 v[11:12], v[12:13], -v[10:11]
; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[8:9]|, s[6:7]
-; SI-SDAG-NEXT: s_mov_b32 s16, 0x509f79ff
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v11, v9, vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v8, v10, v8, vcc
-; SI-SDAG-NEXT: v_mul_f64 v[10:11], v[2:3], s[4:5]
-; SI-SDAG-NEXT: s_mov_b32 s17, 0xbfd34413
-; SI-SDAG-NEXT: v_bfi_b32 v15, s8, v16, v11
-; SI-SDAG-NEXT: v_add_f64 v[12:13], v[10:11], v[14:15]
-; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[10:11]|, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[12:13], v[12:13], -v[14:15]
-; SI-SDAG-NEXT: s_mov_b32 s14, 0xa994fd21
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v11, v13, v11, vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v10, v12, v10, vcc
-; SI-SDAG-NEXT: v_mul_f64 v[12:13], v[4:5], s[4:5]
-; SI-SDAG-NEXT: s_mov_b32 s15, 0x3c49dc1d
-; SI-SDAG-NEXT: v_bfi_b32 v15, s8, v16, v13
-; SI-SDAG-NEXT: v_add_f64 v[17:18], v[12:13], v[14:15]
+; SI-SDAG-NEXT: s_mov_b32 s8, 0x509f79ff
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v12, v9, vcc
+; SI-SDAG-NEXT: v_mul_f64 v[12:13], v[2:3], s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v8, v11, v8, vcc
+; SI-SDAG-NEXT: v_bfi_b32 v11, s10, v16, v13
+; SI-SDAG-NEXT: v_add_f64 v[14:15], v[12:13], v[10:11]
; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[12:13]|, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[17:18], v[17:18], -v[14:15]
-; SI-SDAG-NEXT: s_mov_b32 s20, 0x494ea3e9
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v13, v18, v13, vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v12, v17, v12, vcc
+; SI-SDAG-NEXT: v_add_f64 v[14:15], v[14:15], -v[10:11]
+; SI-SDAG-NEXT: s_mov_b32 s9, 0xbfd34413
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v13, v15, v13, vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v12, v14, v12, vcc
+; SI-SDAG-NEXT: v_mul_f64 v[14:15], v[4:5], s[4:5]
+; SI-SDAG-NEXT: s_mov_b32 s11, 0xbcaf48ad
+; SI-SDAG-NEXT: v_bfi_b32 v11, s10, v16, v15
+; SI-SDAG-NEXT: v_add_f64 v[17:18], v[14:15], v[10:11]
+; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[14:15]|, s[6:7]
+; SI-SDAG-NEXT: v_add_f64 v[17:18], v[17:18], -v[10:11]
+; SI-SDAG-NEXT: s_mov_b32 s12, 0xbbb55516
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v15, v18, v15, vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v14, v17, v14, vcc
; SI-SDAG-NEXT: v_mul_f64 v[17:18], v[6:7], s[4:5]
-; SI-SDAG-NEXT: s_mov_b32 s21, 0xbcaf48ad
-; SI-SDAG-NEXT: v_bfi_b32 v15, s8, v16, v18
-; SI-SDAG-NEXT: v_add_f64 v[19:20], v[17:18], v[14:15]
+; SI-SDAG-NEXT: s_mov_b32 s13, 0x40026bb1
+; SI-SDAG-NEXT: v_bfi_b32 v11, s10, v16, v18
+; SI-SDAG-NEXT: v_add_f64 v[19:20], v[17:18], v[10:11]
; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[17:18]|, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[14:15], v[19:20], -v[14:15]
-; SI-SDAG-NEXT: s_mov_b32 s22, 0xbbb55516
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v14, v14, v17, vcc
-; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], s[16:17], v[0:1]
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v15, v15, v18, vcc
-; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], s[14:15], v[16:17]
-; SI-SDAG-NEXT: s_mov_b32 s23, 0x40026bb1
-; SI-SDAG-NEXT: v_mul_f64 v[18:19], v[16:17], s[20:21]
+; SI-SDAG-NEXT: v_add_f64 v[10:11], v[19:20], -v[10:11]
+; SI-SDAG-NEXT: s_mov_b32 s6, 0xa994fd21
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v10, v10, v17, vcc
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], s[8:9], v[0:1]
+; SI-SDAG-NEXT: s_mov_b32 s7, 0x3c49dc1d
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], s[6:7], v[16:17]
+; SI-SDAG-NEXT: s_mov_b32 s10, 0x494ea3e9
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v11, v11, v18, vcc
+; SI-SDAG-NEXT: v_mul_f64 v[18:19], v[16:17], s[10:11]
; SI-SDAG-NEXT: s_mov_b32 s4, 0xfca7ab0c
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[16:17], s[22:23], v[18:19]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[16:17], s[12:13], v[18:19]
; SI-SDAG-NEXT: s_mov_b32 s5, 0x3e928af3
-; SI-SDAG-NEXT: s_mov_b32 s10, 0x6a5dcb37
-; SI-SDAG-NEXT: s_mov_b32 s11, 0x3e5ade15
-; SI-SDAG-NEXT: v_mov_b32_e32 v17, s5
-; SI-SDAG-NEXT: v_mov_b32_e32 v16, s4
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], s[10:11], v[16:17]
-; SI-SDAG-NEXT: s_mov_b32 s12, 0x623fde64
-; SI-SDAG-NEXT: s_mov_b32 s13, 0x3ec71dee
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[12:13]
+; SI-SDAG-NEXT: s_mov_b32 s14, 0x6a5dcb37
+; SI-SDAG-NEXT: s_mov_b32 s15, 0x3e5ade15
+; SI-SDAG-NEXT: v_mov_b32_e32 v19, s5
+; SI-SDAG-NEXT: v_mov_b32_e32 v18, s4
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], s[14:15], v[18:19]
+; SI-SDAG-NEXT: s_mov_b32 s16, 0x623fde64
+; SI-SDAG-NEXT: s_mov_b32 s17, 0x3ec71dee
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[16:17]
; SI-SDAG-NEXT: s_mov_b32 s18, 0x7c89e6b0
; SI-SDAG-NEXT: s_mov_b32 s19, 0x3efa0199
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[18:19]
-; SI-SDAG-NEXT: s_mov_b32 s24, 0x14761f6e
-; SI-SDAG-NEXT: s_mov_b32 s25, 0x3f2a01a0
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[24:25]
-; SI-SDAG-NEXT: s_mov_b32 s26, 0x1852b7b0
-; SI-SDAG-NEXT: s_mov_b32 s27, 0x3f56c16c
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[26:27]
-; SI-SDAG-NEXT: s_mov_b32 s42, 0x11122322
-; SI-SDAG-NEXT: s_mov_b32 s43, 0x3f811111
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[42:43]
-; SI-SDAG-NEXT: s_mov_b32 s44, 0x555502a1
-; SI-SDAG-NEXT: s_mov_b32 s45, 0x3fa55555
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[44:45]
-; SI-SDAG-NEXT: s_mov_b32 s46, 0x55555511
-; SI-SDAG-NEXT: s_mov_b32 s47, 0x3fc55555
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[46:47]
-; SI-SDAG-NEXT: s_mov_b32 s56, 11
-; SI-SDAG-NEXT: s_mov_b32 s57, 0x3fe00000
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[56:57]
-; SI-SDAG-NEXT: s_mov_b32 s28, 0
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], 1.0
-; SI-SDAG-NEXT: s_mov_b32 s29, 0x40900000
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[18:19], v[20:21], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[10:11], s[16:17], v[2:3]
-; SI-SDAG-NEXT: s_mov_b32 s40, 0
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[10:11], s[14:15], v[20:21]
-; SI-SDAG-NEXT: s_mov_b32 s41, 0xc090cc00
-; SI-SDAG-NEXT: v_mul_f64 v[22:23], v[20:21], s[20:21]
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[4:5], s[28:29], v[0:1]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[20:21], s[22:23], v[22:23]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e32 vcc, s[40:41], v[0:1]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], s[10:11], v[16:17]
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[6:7], s[28:29], v[2:3]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[12:13]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[8:9], s[40:41], v[2:3]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[18:19]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[24:25]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[26:27]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[42:43]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[44:45]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[46:47]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], s[56:57]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[20:21], v[0:1], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[20:21], v[0:1], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[12:13], s[16:17], v[4:5]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[12:13], s[14:15], v[0:1]
-; SI-SDAG-NEXT: v_mul_f64 v[2:3], v[0:1], s[20:21]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[0:1], s[22:23], v[2:3]
-; SI-SDAG-NEXT: v_fma_f64 v[2:3], v[14:15], s[16:17], v[6:7]
-; SI-SDAG-NEXT: v_fma_f64 v[2:3], v[14:15], s[14:15], v[2:3]
-; SI-SDAG-NEXT: s_and_b64 s[14:15], vcc, s[4:5]
-; SI-SDAG-NEXT: v_mul_f64 v[22:23], v[2:3], s[20:21]
-; SI-SDAG-NEXT: v_fma_f64 v[2:3], v[2:3], s[22:23], v[22:23]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], s[10:11], v[16:17]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[12:13]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[18:19]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[24:25]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[26:27]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[42:43]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[44:45]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[46:47]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], s[56:57]
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[22:23], v[0:1], v[22:23], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], s[10:11], v[16:17]
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[10:11], s[28:29], v[4:5]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[12:13]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[12:13], s[40:41], v[4:5]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[18:19]
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v4, v[12:13]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[24:25]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[26:27]
-; SI-SDAG-NEXT: v_ldexp_f64 v[4:5], v[22:23], v4
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[42:43]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[44:45]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[46:47]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], s[56:57]
-; SI-SDAG-NEXT: v_fma_f64 v[0:1], v[2:3], v[0:1], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[2:3], v[0:1], 1.0
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v0, v[8:9]
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v2, v[10:11]
-; SI-SDAG-NEXT: v_mov_b32_e32 v10, 0x7ff00000
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v5, v10, v5, s[10:11]
-; SI-SDAG-NEXT: v_ldexp_f64 v[0:1], v[18:19], v0
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v5, 0, v5, s[12:13]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, v10, v1, s[4:5]
-; SI-SDAG-NEXT: v_ldexp_f64 v[1:2], v[20:21], v2
-; SI-SDAG-NEXT: s_and_b64 s[4:5], s[8:9], s[6:7]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v8, v10, v2, s[6:7]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v2, 0, v1, s[4:5]
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v1, 0, v3, vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, 0, v8, s[8:9]
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v8, v[14:15]
-; SI-SDAG-NEXT: s_and_b64 vcc, s[12:13], s[10:11]
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v4, 0, v4, vcc
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[28:29], v[6:7]
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[18:19]
+; SI-SDAG-NEXT: s_mov_b32 s20, 0x14761f6e
+; SI-SDAG-NEXT: s_mov_b32 s21, 0x3f2a01a0
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[20:21]
+; SI-SDAG-NEXT: s_mov_b32 s22, 0x1852b7b0
+; SI-SDAG-NEXT: s_mov_b32 s23, 0x3f56c16c
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[22:23]
+; SI-SDAG-NEXT: s_mov_b32 s24, 0x11122322
+; SI-SDAG-NEXT: s_mov_b32 s25, 0x3f811111
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[24:25]
+; SI-SDAG-NEXT: s_mov_b32 s26, 0x555502a1
+; SI-SDAG-NEXT: s_mov_b32 s27, 0x3fa55555
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[26:27]
+; SI-SDAG-NEXT: s_mov_b32 s28, 0x55555511
+; SI-SDAG-NEXT: s_mov_b32 s29, 0x3fc55555
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[28:29]
+; SI-SDAG-NEXT: s_mov_b32 s40, 11
+; SI-SDAG-NEXT: s_mov_b32 s41, 0x3fe00000
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], s[40:41]
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[16:17], v[20:21], 1.0
+; SI-SDAG-NEXT: s_mov_b32 s42, 0
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[16:17], v[20:21], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[12:13], s[8:9], v[2:3]
+; SI-SDAG-NEXT: s_mov_b32 s43, 0x40900000
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[12:13], s[6:7], v[20:21]
+; SI-SDAG-NEXT: s_mov_b32 s44, 0
+; SI-SDAG-NEXT: v_mul_f64 v[22:23], v[20:21], s[10:11]
+; SI-SDAG-NEXT: s_mov_b32 s45, 0xc090cc00
; SI-SDAG-NEXT: v_ldexp_f64 v[8:9], v[16:17], v8
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[40:41], v[6:7]
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v6, v10, v9, vcc
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[42:43], v[0:1]
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[0:1]
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[20:21], s[12:13], v[22:23]
+; SI-SDAG-NEXT: v_mov_b32_e32 v22, 0x7ff00000
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v22, v9, vcc
+; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v1, 0, v9, s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v0, 0, v8, vcc
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], s[14:15], v[18:19]
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v16, v[12:13]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[16:17]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[14:15], s[8:9], v[4:5]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[18:19]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[14:15], s[6:7], v[12:13]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[20:21]
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[42:43], v[2:3]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[22:23]
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[2:3]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[24:25]
+; SI-SDAG-NEXT: v_mul_f64 v[2:3], v[12:13], s[10:11]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[26:27]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[12:13], s[12:13], v[2:3]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[28:29]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], s[40:41]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], 1.0
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v20, v[14:15]
+; SI-SDAG-NEXT: v_ldexp_f64 v[8:9], v[8:9], v16
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[10:11], s[8:9], v[6:7]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v22, v9, vcc
; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v0, 0, v0, s[14:15]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v7, 0, v6, s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, 0, v9, s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v2, 0, v8, vcc
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[10:11], s[6:7], v[16:17]
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v10, v[10:11]
+; SI-SDAG-NEXT: v_mul_f64 v[14:15], v[8:9], s[10:11]
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[42:43], v[4:5]
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[8:9], s[12:13], v[14:15]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], s[14:15], v[18:19]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], s[14:15], v[18:19]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[16:17]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[16:17]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[18:19]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[18:19]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[20:21]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[20:21]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[22:23]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[22:23]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[24:25]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[24:25]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[26:27]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[26:27]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[28:29]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[28:29]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], s[40:41]
+; SI-SDAG-NEXT: v_fma_f64 v[16:17], v[8:9], v[16:17], s[40:41]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[12:13], v[14:15], 1.0
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[4:5]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[12:13], v[14:15], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[8:9], v[16:17], 1.0
+; SI-SDAG-NEXT: v_ldexp_f64 v[12:13], v[12:13], v20
+; SI-SDAG-NEXT: v_fma_f64 v[8:9], v[8:9], v[14:15], 1.0
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v11, v22, v13, vcc
+; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v5, 0, v11, s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v4, 0, v12, vcc
+; SI-SDAG-NEXT: v_ldexp_f64 v[8:9], v[8:9], v10
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[42:43], v[6:7]
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[6:7]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v22, v9, vcc
+; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v7, 0, v9, s[4:5]
; SI-SDAG-NEXT: v_cndmask_b32_e32 v6, 0, v8, vcc
; SI-SDAG-NEXT: s_setpc_b64 s[30:31]
;
@@ -1917,160 +1917,160 @@ define <4 x double> @v_exp10_v4f64(<4 x double> %in) #0 {
; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0
; SI-GISEL-NEXT: v_and_b32_e32 v10, 0x80000000, v9
; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v10
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[8:9], v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v10, -1
-; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x432fffff
-; SI-GISEL-NEXT: v_add_f64 v[13:14], v[14:15], -v[12:13]
+; SI-GISEL-NEXT: v_add_f64 v[10:11], v[8:9], v[12:13]
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, -1
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x432fffff
+; SI-GISEL-NEXT: v_add_f64 v[10:11], v[10:11], -v[12:13]
; SI-GISEL-NEXT: v_mul_f64 v[18:19], v[2:3], v[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[8:9]|, v[10:11]
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0xbbb55516
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v13, v8, vcc
-; SI-GISEL-NEXT: v_and_b32_e32 v13, 0x80000000, v19
-; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v13
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[8:9]|, v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0xa994fd21
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v10, v8, vcc
+; SI-GISEL-NEXT: v_and_b32_e32 v10, 0x80000000, v19
+; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v10
; SI-GISEL-NEXT: v_add_f64 v[20:21], v[18:19], v[12:13]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v14, v9, vcc
-; SI-GISEL-NEXT: v_add_f64 v[14:15], v[20:21], -v[12:13]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v11, v9, vcc
+; SI-GISEL-NEXT: v_add_f64 v[10:11], v[20:21], -v[12:13]
; SI-GISEL-NEXT: v_mul_f64 v[20:21], v[4:5], v[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[18:19]|, v[10:11]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[18:19]|, v[14:15]
; SI-GISEL-NEXT: v_and_b32_e32 v13, 0x80000000, v21
; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v13
; SI-GISEL-NEXT: v_add_f64 v[22:23], v[20:21], v[12:13]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v14, v14, v18, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v15, v15, v19, vcc
+; SI-GISEL-NEXT: v_mul_f64 v[16:17], v[6:7], v[16:17]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, v10, v18, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v11, v19, vcc
; SI-GISEL-NEXT: v_add_f64 v[18:19], v[22:23], -v[12:13]
-; SI-GISEL-NEXT: v_mul_f64 v[22:23], v[6:7], v[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[20:21]|, v[10:11]
-; SI-GISEL-NEXT: v_and_b32_e32 v13, 0x80000000, v23
+; SI-GISEL-NEXT: v_and_b32_e32 v13, 0x80000000, v17
; SI-GISEL-NEXT: v_or_b32_e32 v13, 0x43300000, v13
-; SI-GISEL-NEXT: v_add_f64 v[24:25], v[22:23], v[12:13]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v18, v20, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v19, v21, vcc
-; SI-GISEL-NEXT: v_add_f64 v[12:13], v[24:25], -v[12:13]
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0x509f79ff
-; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x3fd34413
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[22:23]|, v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], -v[8:9], v[18:19], v[0:1]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, v12, v22, vcc
-; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0xa994fd21
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[20:21]|, v[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[22:23], v[16:17], v[12:13]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v18, v18, v20, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v19, v19, v21, vcc
+; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x509f79ff
+; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x3fd34413
+; SI-GISEL-NEXT: v_add_f64 v[12:13], v[22:23], -v[12:13]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[16:17]|, v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[8:9], v[20:21], v[0:1]
; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0xbc49dc1d
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v13, v23, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[14:15], v[18:19], v[2:3]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], -v[8:9], v[24:25], v[20:21]
-; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x494ea3e9
-; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0xbcaf48ad
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], -v[14:15], v[24:25], v[22:23]
-; SI-GISEL-NEXT: v_mul_f64 v[26:27], v[12:13], v[20:21]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v12, v12, v16, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v13, v13, v17, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], -v[10:11], v[20:21], v[2:3]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], -v[8:9], v[24:25], v[22:23]
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0x494ea3e9
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0xbcaf48ad
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], -v[10:11], v[24:25], v[16:17]
+; SI-GISEL-NEXT: v_mul_f64 v[26:27], v[14:15], v[22:23]
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0xbbb55516
; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x40026bb1
-; SI-GISEL-NEXT: v_mul_f64 v[30:31], v[22:23], v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[12:13], v[28:29], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[22:23], v[28:29], v[30:31]
+; SI-GISEL-NEXT: v_mul_f64 v[30:31], v[16:17], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[14:15], v[28:29], v[26:27]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[16:17], v[28:29], v[30:31]
; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x6a5dcb37
; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3e5ade15
; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0xfca7ab0c
; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3e928af3
-; SI-GISEL-NEXT: v_fma_f64 v[26:27], -v[16:17], v[18:19], v[4:5]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], -v[10:11], v[18:19], v[6:7]
-; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[12:13], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], -v[18:19], v[20:21], v[4:5]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], -v[12:13], v[20:21], v[6:7]
+; SI-GISEL-NEXT: v_fma_f64 v[34:35], v[14:15], v[30:31], v[32:33]
; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x623fde64
; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3ec71dee
-; SI-GISEL-NEXT: v_fma_f64 v[26:27], -v[16:17], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], -v[10:11], v[24:25], v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[34:35], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], -v[18:19], v[24:25], v[26:27]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], -v[12:13], v[24:25], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[14:15], v[34:35], v[36:37]
; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x7c89e6b0
; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3efa0199
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[34:35]
-; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[26:27], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[14:15], v[24:25], v[34:35]
+; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[26:27], v[22:23]
; SI-GISEL-NEXT: v_mov_b32_e32 v48, 0x14761f6e
; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3f2a01a0
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[14:15], v[24:25], v[48:49]
; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[26:27], v[28:29], v[38:39]
; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x1852b7b0
; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3f56c16c
-; SI-GISEL-NEXT: v_mul_f64 v[20:21], v[18:19], v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[38:39]
+; SI-GISEL-NEXT: v_mul_f64 v[22:23], v[20:21], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[14:15], v[24:25], v[38:39]
; SI-GISEL-NEXT: v_mov_b32_e32 v50, 0x11122322
; SI-GISEL-NEXT: v_mov_b32_e32 v51, 0x3f811111
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[18:19], v[28:29], v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[12:13], v[24:25], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[20:21], v[28:29], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[24:25], v[50:51]
; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0x555502a1
; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3fa55555
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[12:13], v[20:21], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[24:25]
; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x55555511
; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3fc55555
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[12:13], v[20:21], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[28:29]
; SI-GISEL-NEXT: v_mov_b32_e32 v52, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v53, 0x3fe00000
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[12:13], v[20:21], v[52:53]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[12:13], v[20:21], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v10, v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[12:13], v[20:21], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[26:27], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[12:13], v8
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[24:25]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], v[52:53]
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[22:23], v[12:13], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[18:19], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v32, v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[20:21], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[22:23], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[38:39]
-; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[50:51]
-; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x40900000
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[24:25]
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], v[52:53]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v33, 0, v8, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[16:17]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[24:25]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[26:27], v[14:15], 1.0
-; SI-GISEL-NEXT: v_mov_b32_e32 v16, 0x7ff00000
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[28:29]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v16, v9, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[14:15], v8
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[30:31]
-; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0xc090cc00
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], v[52:53]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v14, 0, v8, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v15, v16, v9, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[22:23]
-; SI-GISEL-NEXT: v_ldexp_f64 v[12:13], v[12:13], v32
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e64 s[4:5], v[2:3], v[30:31]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v33, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v17, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[18:19], v[20:21], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e64 v12, 0, v12, s[4:5]
-; SI-GISEL-NEXT: v_cndmask_b32_e64 v13, v16, v13, s[4:5]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v12, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v13, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[18:19], v[8:9], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v14, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v15, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v10
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[6:7], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[52:53]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v18, v[18:19]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], 1.0
+; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x7ff00000
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[14:15], v[22:23], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v12, v[12:13]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], v[52:53]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[16:17], v[22:23], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[16:17], v[22:23], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[22:23], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[22:23], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[22:23], v[48:49]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v36, v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[22:23], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[22:23], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[26:27], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[20:21], v[30:31], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[26:27], v[8:9], v[28:29]
+; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[26:27], v[8:9], v[52:53]
+; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x40900000
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[26:27], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v37, v[10:11]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[30:31], v[24:25]
+; SI-GISEL-NEXT: v_ldexp_f64 v[14:15], v[14:15], v36
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[26:27], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v14, 0, v14, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], v[28:29]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v15, v19, v15, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v18
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[32:33]
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0xc090cc00
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[20:21], v[10:11], v[52:53]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v18, 0, v8, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v22, v19, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[34:35]
+; SI-GISEL-NEXT: v_ldexp_f64 v[16:17], v[16:17], v37
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e64 s[4:5], v[2:3], v[32:33]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v14, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v15, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[10:11], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e64 v16, 0, v16, s[4:5]
+; SI-GISEL-NEXT: v_cndmask_b32_e64 v17, v19, v17, s[4:5]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v16, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v17, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[34:35]
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v12
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v18, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v22, vcc
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[6:7], v[32:33]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, 0, v8, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[6:7], v[22:23]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v19, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[6:7], v[34:35]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v8, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, 0, v9, vcc
; SI-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -3931,12 +3931,15 @@ define amdgpu_ps <6 x i32> @s_exp10_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_mov_b32_e32 v34, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3fe00000
; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[4:5], v[18:19], v[34:35]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v36, s[10:11]
+; SI-GISEL-NEXT: v_mul_f64 v[36:37], s[2:3], v[0:1]
; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[4:5], v[18:19], 1.0
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[36:37]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[4:5], v[4:5], v[18:19], 1.0
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0
-; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v36
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v18, s[10:11]
; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x40900000
+; SI-GISEL-NEXT: v_mul_f64 v[0:1], s[4:5], v[0:1]
+; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v18
+; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[0:1], v[18:19]
; SI-GISEL-NEXT: v_readfirstlane_b32 s12, v4
; SI-GISEL-NEXT: v_readfirstlane_b32 s13, v5
@@ -3946,7 +3949,6 @@ define amdgpu_ps <6 x i32> @s_exp10_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[0:1], v[4:5]
; SI-GISEL-NEXT: s_mov_b32 s10, 0
; SI-GISEL-NEXT: s_mov_b32 s11, 0x7ff00000
-; SI-GISEL-NEXT: v_mul_f64 v[36:37], s[2:3], v[0:1]
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[12:13], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[12:13], vcc, vcc
; SI-GISEL-NEXT: v_readfirstlane_b32 s12, v36
@@ -3954,24 +3956,22 @@ define amdgpu_ps <6 x i32> @s_exp10_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0
; SI-GISEL-NEXT: s_and_b64 s[14:15], s[12:13], s[8:9]
; SI-GISEL-NEXT: s_or_b64 s[14:15], s[14:15], s[6:7]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[36:37], s[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], s[14:15]
+; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], -s[14:15]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[2:3], v[18:19]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[38:39], -s[14:15]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[14:15], |v[36:37]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s16, v38
-; SI-GISEL-NEXT: v_readfirstlane_b32 s17, v39
-; SI-GISEL-NEXT: s_or_b64 s[14:15], s[14:15], s[14:15]
-; SI-GISEL-NEXT: s_cselect_b64 s[12:13], s[12:13], s[16:17]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v36
+; SI-GISEL-NEXT: v_readfirstlane_b32 s15, v37
+; SI-GISEL-NEXT: s_cselect_b64 s[12:13], s[12:13], s[14:15]
; SI-GISEL-NEXT: v_mov_b32_e32 v37, s13
; SI-GISEL-NEXT: v_mov_b32_e32 v36, s12
; SI-GISEL-NEXT: v_fma_f64 v[36:37], -v[36:37], v[2:3], s[2:3]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v40, s[12:13]
+; SI-GISEL-NEXT: s_or_b64 s[16:17], vcc, vcc
; SI-GISEL-NEXT: v_fma_f64 v[36:37], -s[12:13], v[8:9], v[36:37]
-; SI-GISEL-NEXT: s_or_b64 s[14:15], vcc, vcc
-; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[36:37], v[10:11]
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[2:3], v[4:5]
+; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[36:37], v[10:11]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[14:15], |v[0:1]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[14:15], v[38:39]
-; SI-GISEL-NEXT: v_mul_f64 v[0:1], s[4:5], v[0:1]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[12:13], v[16:17]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[20:21]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[22:23]
@@ -3983,7 +3983,8 @@ define amdgpu_ps <6 x i32> @s_exp10_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[34:35]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], 1.0
; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[38:39], 1.0
-; SI-GISEL-NEXT: v_ldexp_f64 v[36:37], v[36:37], v40
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v38, s[12:13]
+; SI-GISEL-NEXT: v_ldexp_f64 v[36:37], v[36:37], v38
; SI-GISEL-NEXT: v_readfirstlane_b32 s12, v36
; SI-GISEL-NEXT: v_readfirstlane_b32 s13, v37
; SI-GISEL-NEXT: s_cselect_b64 s[2:3], s[12:13], s[10:11]
@@ -3993,14 +3994,13 @@ define amdgpu_ps <6 x i32> @s_exp10_v3f64(<3 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[2:3], s[2:3], 0
; SI-GISEL-NEXT: s_and_b64 s[8:9], s[12:13], s[8:9]
; SI-GISEL-NEXT: s_or_b64 s[6:7], s[8:9], s[6:7]
-; SI-GISEL-NEXT: v_add_f64 v[36:37], v[0:1], s[6:7]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], s[6:7]
+; SI-GISEL-NEXT: s_or_b64 s[8:9], s[14:15], s[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], -s[6:7]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[4:5], v[18:19]
-; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], -s[6:7]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[6:7], |v[0:1]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s8, v36
-; SI-GISEL-NEXT: v_readfirstlane_b32 s9, v37
-; SI-GISEL-NEXT: s_or_b64 s[6:7], s[6:7], s[6:7]
-; SI-GISEL-NEXT: s_cselect_b64 s[6:7], s[12:13], s[8:9]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s6, v0
+; SI-GISEL-NEXT: v_readfirstlane_b32 s7, v1
+; SI-GISEL-NEXT: s_cselect_b64 s[6:7], s[12:13], s[6:7]
; SI-GISEL-NEXT: v_mov_b32_e32 v0, s6
; SI-GISEL-NEXT: v_mov_b32_e32 v1, s7
; SI-GISEL-NEXT: v_fma_f64 v[0:1], -v[0:1], v[2:3], s[4:5]
@@ -4785,12 +4785,14 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_mov_b32_e32 v34, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3fe00000
; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[4:5], v[18:19], v[34:35]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v36, s[12:13]
+; SI-GISEL-NEXT: v_mul_f64 v[36:37], s[2:3], v[0:1]
; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[4:5], v[18:19], 1.0
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[18:19], |v[36:37]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[4:5], v[4:5], v[18:19], 1.0
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0
-; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v36
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v18, s[12:13]
; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x40900000
+; SI-GISEL-NEXT: v_ldexp_f64 v[4:5], v[4:5], v18
+; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[0:1], v[18:19]
; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v4
; SI-GISEL-NEXT: v_readfirstlane_b32 s15, v5
@@ -4800,7 +4802,6 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[0:1], v[4:5]
; SI-GISEL-NEXT: s_mov_b32 s12, 0
; SI-GISEL-NEXT: s_mov_b32 s13, 0x7ff00000
-; SI-GISEL-NEXT: v_mul_f64 v[36:37], s[2:3], v[0:1]
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[14:15], s[12:13]
; SI-GISEL-NEXT: s_or_b64 s[14:15], vcc, vcc
; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v36
@@ -4808,22 +4809,20 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0
; SI-GISEL-NEXT: s_and_b64 s[16:17], s[14:15], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[36:37], s[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], s[16:17]
+; SI-GISEL-NEXT: s_or_b64 s[18:19], s[18:19], s[18:19]
+; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], -s[16:17]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[2:3], v[18:19]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[38:39], -s[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[36:37]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s18, v38
-; SI-GISEL-NEXT: v_readfirstlane_b32 s19, v39
-; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[16:17]
-; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[18:19]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s16, v36
+; SI-GISEL-NEXT: v_readfirstlane_b32 s17, v37
+; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[16:17]
; SI-GISEL-NEXT: v_mov_b32_e32 v37, s15
; SI-GISEL-NEXT: v_mov_b32_e32 v36, s14
; SI-GISEL-NEXT: v_fma_f64 v[36:37], -v[36:37], v[2:3], s[2:3]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v40, s[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[36:37], -s[14:15], v[8:9], v[36:37]
; SI-GISEL-NEXT: s_or_b64 s[16:17], vcc, vcc
-; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[36:37], v[10:11]
+; SI-GISEL-NEXT: v_fma_f64 v[36:37], -s[14:15], v[8:9], v[36:37]
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[2:3], v[4:5]
+; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[36:37], v[10:11]
; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[14:15], v[38:39]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[12:13], v[16:17]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[20:21]
@@ -4836,7 +4835,8 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[34:35]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], 1.0
; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[38:39], 1.0
-; SI-GISEL-NEXT: v_ldexp_f64 v[36:37], v[36:37], v40
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v38, s[14:15]
+; SI-GISEL-NEXT: v_ldexp_f64 v[36:37], v[36:37], v38
; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v36
; SI-GISEL-NEXT: v_readfirstlane_b32 s15, v37
; SI-GISEL-NEXT: v_mul_f64 v[36:37], s[4:5], v[0:1]
@@ -4847,24 +4847,24 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[2:3], s[2:3], 0
; SI-GISEL-NEXT: s_and_b64 s[16:17], s[14:15], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[36:37], s[16:17]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[18:19], |v[36:37]|, v[6:7]
+; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], s[16:17]
+; SI-GISEL-NEXT: s_or_b64 s[18:19], s[18:19], s[18:19]
+; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], -s[16:17]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[4:5], v[18:19]
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[38:39], -s[16:17]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[36:37]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s18, v38
-; SI-GISEL-NEXT: v_readfirstlane_b32 s19, v39
-; SI-GISEL-NEXT: s_or_b64 s[16:17], s[16:17], s[16:17]
-; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[18:19]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s16, v36
+; SI-GISEL-NEXT: v_readfirstlane_b32 s17, v37
+; SI-GISEL-NEXT: s_cselect_b64 s[14:15], s[14:15], s[16:17]
; SI-GISEL-NEXT: v_mov_b32_e32 v37, s15
; SI-GISEL-NEXT: v_mov_b32_e32 v36, s14
; SI-GISEL-NEXT: v_fma_f64 v[36:37], -v[36:37], v[2:3], s[4:5]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v40, s[14:15]
+; SI-GISEL-NEXT: s_or_b64 s[18:19], vcc, vcc
; SI-GISEL-NEXT: v_fma_f64 v[36:37], -s[14:15], v[8:9], v[36:37]
-; SI-GISEL-NEXT: s_or_b64 s[16:17], vcc, vcc
-; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[36:37], v[10:11]
; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, s[4:5], v[4:5]
-; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[14:15], v[38:39]
+; SI-GISEL-NEXT: v_mul_f64 v[38:39], v[36:37], v[10:11]
; SI-GISEL-NEXT: v_mul_f64 v[0:1], s[6:7], v[0:1]
+; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[14:15], v[38:39]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[16:17], |v[0:1]|, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[12:13], v[16:17]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[20:21]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[22:23]
@@ -4876,7 +4876,8 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], v[34:35]
; SI-GISEL-NEXT: v_fma_f64 v[38:39], v[36:37], v[38:39], 1.0
; SI-GISEL-NEXT: v_fma_f64 v[36:37], v[36:37], v[38:39], 1.0
-; SI-GISEL-NEXT: v_ldexp_f64 v[36:37], v[36:37], v40
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v38, s[14:15]
+; SI-GISEL-NEXT: v_ldexp_f64 v[36:37], v[36:37], v38
; SI-GISEL-NEXT: v_readfirstlane_b32 s14, v36
; SI-GISEL-NEXT: v_readfirstlane_b32 s15, v37
; SI-GISEL-NEXT: s_cselect_b64 s[4:5], s[14:15], s[12:13]
@@ -4886,14 +4887,13 @@ define amdgpu_ps <8 x i32> @s_exp10_v4f64(<4 x double> inreg %in) #0 {
; SI-GISEL-NEXT: s_cselect_b64 s[4:5], s[4:5], 0
; SI-GISEL-NEXT: s_and_b64 s[10:11], s[14:15], s[10:11]
; SI-GISEL-NEXT: s_or_b64 s[8:9], s[10:11], s[8:9]
-; SI-GISEL-NEXT: v_add_f64 v[36:37], v[0:1], s[8:9]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], s[8:9]
+; SI-GISEL-NEXT: s_or_b64 s[10:11], s[16:17], s[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[0:1], v[0:1], -s[8:9]
; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, s[6:7], v[18:19]
-; SI-GISEL-NEXT: v_add_f64 v[36:37], v[36:37], -s[8:9]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 s[8:9], |v[0:1]|, v[6:7]
-; SI-GISEL-NEXT: v_readfirstlane_b32 s10, v36
-; SI-GISEL-NEXT: v_readfirstlane_b32 s11, v37
-; SI-GISEL-NEXT: s_or_b64 s[8:9], s[8:9], s[8:9]
-; SI-GISEL-NEXT: s_cselect_b64 s[8:9], s[14:15], s[10:11]
+; SI-GISEL-NEXT: v_readfirstlane_b32 s8, v0
+; SI-GISEL-NEXT: v_readfirstlane_b32 s9, v1
+; SI-GISEL-NEXT: s_cselect_b64 s[8:9], s[14:15], s[8:9]
; SI-GISEL-NEXT: v_mov_b32_e32 v0, s8
; SI-GISEL-NEXT: v_mov_b32_e32 v1, s9
; SI-GISEL-NEXT: v_fma_f64 v[0:1], -v[0:1], v[2:3], s[6:7]
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.exp2.f64.ll b/llvm/test/CodeGen/AMDGPU/llvm.exp2.f64.ll
index 69bf598973247..86deda74e4370 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.exp2.f64.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.exp2.f64.ll
@@ -1025,118 +1025,118 @@ define <3 x double> @v_exp2_v3f64(<3 x double> %in) #0 {
; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v6
; SI-GISEL-NEXT: v_add_f64 v[6:7], v[0:1], v[8:9]
; SI-GISEL-NEXT: v_mov_b32_e32 v10, -1
-; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x432fffff
; SI-GISEL-NEXT: v_add_f64 v[6:7], v[6:7], -v[8:9]
; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v3
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[0:1]|, v[10:11]
+; SI-GISEL-NEXT: v_mov_b32_e32 v11, 0x432fffff
; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[0:1]|, v[10:11]
; SI-GISEL-NEXT: v_add_f64 v[18:19], v[2:3], v[8:9]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, v6, v0, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v7, v1, vcc
-; SI-GISEL-NEXT: v_add_f64 v[12:13], v[0:1], -v[6:7]
; SI-GISEL-NEXT: v_add_f64 v[18:19], v[18:19], -v[8:9]
-; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v5
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[2:3]|, v[10:11]
+; SI-GISEL-NEXT: v_add_f64 v[12:13], v[0:1], -v[6:7]
; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0x3b39803f
; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x3c7abc9e
-; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v18, v18, v2, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v19, v19, v3, vcc
; SI-GISEL-NEXT: v_mul_f64 v[16:17], v[12:13], v[14:15]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[2:3]|, v[10:11]
-; SI-GISEL-NEXT: v_add_f64 v[28:29], v[4:5], v[8:9]
+; SI-GISEL-NEXT: v_add_f64 v[22:23], v[2:3], -v[18:19]
+; SI-GISEL-NEXT: v_and_b32_e32 v9, 0x80000000, v5
; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0xfefa39ef
; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x3fe62e42
+; SI-GISEL-NEXT: v_or_b32_e32 v9, 0x43300000, v9
; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[12:13], v[20:21], v[16:17]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v18, v2, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v19, v3, vcc
-; SI-GISEL-NEXT: v_add_f64 v[8:9], v[28:29], -v[8:9]
+; SI-GISEL-NEXT: v_mul_f64 v[16:17], v[22:23], v[14:15]
+; SI-GISEL-NEXT: v_add_f64 v[24:25], v[4:5], v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[22:23], v[20:21], v[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[8:9], v[24:25], -v[8:9]
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0x6a5dcb37
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x3e5ade15
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0xfca7ab0c
+; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3e928af3
; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[4:5]|, v[10:11]
-; SI-GISEL-NEXT: v_add_f64 v[32:33], v[2:3], -v[16:17]
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0x6a5dcb37
-; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x3e5ade15
-; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0xfca7ab0c
-; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x3e928af3
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v8, v4, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v9, v5, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[18:19], v[22:23]
-; SI-GISEL-NEXT: v_mul_f64 v[36:37], v[32:33], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[50:51], v[4:5], -v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[22:23], v[24:25]
; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0x623fde64
; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0x3ec71dee
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[32:33], v[32:33], v[20:21], v[36:37]
-; SI-GISEL-NEXT: v_mul_f64 v[14:15], v[50:51], v[14:15]
-; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x7c89e6b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3efa0199
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[50:51], v[20:21], v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[50:51], v[32:33], v[18:19], v[22:23]
-; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x14761f6e
-; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3f2a01a0
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[22:23]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[32:33], v[50:51], v[26:27]
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x1852b7b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3f56c16c
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[32:33], v[22:23], v[30:31]
-; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x11122322
-; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3f811111
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[24:25], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[32:33], v[22:23], v[34:35]
-; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0x555502a1
-; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0x3fa55555
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[24:25]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[32:33], v[22:23], v[28:29]
-; SI-GISEL-NEXT: v_mov_b32_e32 v48, 0x55555511
-; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3fc55555
-; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[30:31]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v26, v[6:7]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[32:33], v[22:23], v[38:39]
-; SI-GISEL-NEXT: v_mov_b32_e32 v36, 11
-; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3fe00000
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[26:27]
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x7c89e6b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3efa0199
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v8, v4, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v9, v5, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[28:29]
+; SI-GISEL-NEXT: v_add_f64 v[30:31], v[4:5], -v[8:9]
+; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0x14761f6e
+; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3f2a01a0
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[32:33]
+; SI-GISEL-NEXT: v_mul_f64 v[14:15], v[30:31], v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x1852b7b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3f56c16c
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[30:31], v[20:21], v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0x11122322
+; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0x3f811111
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[20:21]
+; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x555502a1
+; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3fa55555
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[30:31]
+; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x55555511
+; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3fc55555
; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[32:33], v[6:7], v[24:25]
+; SI-GISEL-NEXT: v_mov_b32_e32 v38, 11
+; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3fe00000
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[38:39]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v6, v[6:7]
; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[32:33], v[6:7], v[48:49]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[32:33], v[6:7], v[36:37]
-; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x40900000
-; SI-GISEL-NEXT: v_ldexp_f64 v[10:11], v[10:11], v26
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[12:13]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[24:25]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[32:33], v[6:7], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v16, v[16:17]
-; SI-GISEL-NEXT: v_mov_b32_e32 v20, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v21, 0xc090cc00
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[16:17], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[10:11], v6
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[12:13], v[26:27]
+; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[14:15], v[22:23], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[22:23], v[26:27]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[36:37]
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], v[38:39]
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x40900000
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], 1.0
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[30:31]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[16:17], v[10:11], 1.0
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v16, v[18:19]
+; SI-GISEL-NEXT: v_mov_b32_e32 v24, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v25, 0xc090cc00
; SI-GISEL-NEXT: v_mov_b32_e32 v17, 0x7ff00000
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, 0, v10, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[32:33], v[6:7], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v17, v11, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], v[36:37]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v10, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v16
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v11, vcc
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[2:3], v[12:13]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v6, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[36:37]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v17, v7, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[24:25]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[14:15], v[12:13], v[38:39]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v6, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v7, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[10:11], v16
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[2:3], v[22:23]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, 0, v6, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v17, v7, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[14:15], v[18:19], 1.0
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[14:15], v[12:13], 1.0
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[24:25]
; SI-GISEL-NEXT: v_fma_f64 v[6:7], v[14:15], v[6:7], 1.0
; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v10, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v8
; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v11, vcc
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[12:13]
+; SI-GISEL-NEXT: v_ldexp_f64 v[6:7], v[6:7], v8
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[22:23]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v6, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, v17, v7, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[20:21]
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[24:25]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v6, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v7, vcc
; SI-GISEL-NEXT: s_setpc_b64 s[30:31]
@@ -1564,305 +1564,305 @@ define <4 x double> @v_exp2_v4f64(<4 x double> %in) #0 {
; SI-SDAG-LABEL: v_exp2_v4f64:
; SI-SDAG: ; %bb.0:
; SI-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SI-SDAG-NEXT: s_brev_b32 s62, -2
-; SI-SDAG-NEXT: v_mov_b32_e32 v17, 0x43300000
-; SI-SDAG-NEXT: v_bfi_b32 v11, s62, v17, v1
-; SI-SDAG-NEXT: v_mov_b32_e32 v10, 0
-; SI-SDAG-NEXT: v_add_f64 v[8:9], v[0:1], v[10:11]
-; SI-SDAG-NEXT: s_mov_b32 s14, -1
-; SI-SDAG-NEXT: s_mov_b32 s15, 0x432fffff
-; SI-SDAG-NEXT: v_add_f64 v[8:9], v[8:9], -v[10:11]
-; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[0:1]|, s[14:15]
-; SI-SDAG-NEXT: s_mov_b32 s18, 0x3b39803f
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v9, v1, vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v8, v8, v0, vcc
-; SI-SDAG-NEXT: v_add_f64 v[11:12], v[0:1], -v[8:9]
-; SI-SDAG-NEXT: s_mov_b32 s19, 0x3c7abc9e
-; SI-SDAG-NEXT: v_mul_f64 v[13:14], v[11:12], s[18:19]
-; SI-SDAG-NEXT: s_mov_b32 s20, 0xfefa39ef
-; SI-SDAG-NEXT: s_mov_b32 s21, 0x3fe62e42
-; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[11:12], s[20:21], v[13:14]
+; SI-SDAG-NEXT: s_brev_b32 s56, -2
+; SI-SDAG-NEXT: v_mov_b32_e32 v14, 0x43300000
+; SI-SDAG-NEXT: v_bfi_b32 v9, s56, v14, v1
+; SI-SDAG-NEXT: v_mov_b32_e32 v8, 0
+; SI-SDAG-NEXT: v_add_f64 v[10:11], v[0:1], v[8:9]
+; SI-SDAG-NEXT: s_mov_b32 s10, -1
+; SI-SDAG-NEXT: s_mov_b32 s11, 0x432fffff
+; SI-SDAG-NEXT: v_add_f64 v[9:10], v[10:11], -v[8:9]
+; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[0:1]|, s[10:11]
+; SI-SDAG-NEXT: s_mov_b32 s12, 0x3b39803f
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v11, v10, v1, vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v10, v9, v0, vcc
+; SI-SDAG-NEXT: v_add_f64 v[12:13], v[0:1], -v[10:11]
+; SI-SDAG-NEXT: s_mov_b32 s13, 0x3c7abc9e
+; SI-SDAG-NEXT: v_bfi_b32 v9, s56, v14, v3
+; SI-SDAG-NEXT: v_mul_f64 v[15:16], v[12:13], s[12:13]
+; SI-SDAG-NEXT: v_add_f64 v[17:18], v[2:3], v[8:9]
+; SI-SDAG-NEXT: s_mov_b32 s14, 0xfefa39ef
+; SI-SDAG-NEXT: s_mov_b32 s15, 0x3fe62e42
+; SI-SDAG-NEXT: v_fma_f64 v[15:16], v[12:13], s[14:15], v[15:16]
+; SI-SDAG-NEXT: v_add_f64 v[12:13], v[17:18], -v[8:9]
+; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[2:3]|, s[10:11]
; SI-SDAG-NEXT: s_mov_b32 s4, 0xfca7ab0c
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v18, v13, v3, vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v17, v12, v2, vcc
+; SI-SDAG-NEXT: v_add_f64 v[12:13], v[2:3], -v[17:18]
; SI-SDAG-NEXT: s_mov_b32 s5, 0x3e928af3
-; SI-SDAG-NEXT: s_mov_b32 s22, 0x6a5dcb37
-; SI-SDAG-NEXT: s_mov_b32 s23, 0x3e5ade15
+; SI-SDAG-NEXT: v_mul_f64 v[19:20], v[12:13], s[12:13]
+; SI-SDAG-NEXT: s_mov_b32 s18, 0x6a5dcb37
+; SI-SDAG-NEXT: v_fma_f64 v[19:20], v[12:13], s[14:15], v[19:20]
+; SI-SDAG-NEXT: s_mov_b32 s19, 0x3e5ade15
; SI-SDAG-NEXT: v_mov_b32_e32 v13, s5
; SI-SDAG-NEXT: v_mov_b32_e32 v12, s4
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], s[22:23], v[12:13]
-; SI-SDAG-NEXT: s_mov_b32 s24, 0x623fde64
-; SI-SDAG-NEXT: s_mov_b32 s25, 0x3ec71dee
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[24:25]
-; SI-SDAG-NEXT: s_mov_b32 s26, 0x7c89e6b0
-; SI-SDAG-NEXT: s_mov_b32 s27, 0x3efa0199
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[26:27]
-; SI-SDAG-NEXT: s_mov_b32 s28, 0x14761f6e
-; SI-SDAG-NEXT: s_mov_b32 s29, 0x3f2a01a0
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[28:29]
-; SI-SDAG-NEXT: s_mov_b32 s40, 0x1852b7b0
-; SI-SDAG-NEXT: s_mov_b32 s41, 0x3f56c16c
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[40:41]
-; SI-SDAG-NEXT: s_mov_b32 s42, 0x11122322
-; SI-SDAG-NEXT: s_mov_b32 s43, 0x3f811111
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[42:43]
-; SI-SDAG-NEXT: s_mov_b32 s44, 0x555502a1
-; SI-SDAG-NEXT: s_mov_b32 s45, 0x3fa55555
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[44:45]
-; SI-SDAG-NEXT: s_mov_b32 s46, 0x55555511
-; SI-SDAG-NEXT: s_mov_b32 s47, 0x3fc55555
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[46:47]
-; SI-SDAG-NEXT: s_mov_b32 s56, 11
-; SI-SDAG-NEXT: s_mov_b32 s57, 0x3fe00000
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], s[56:57]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], s[18:19], v[12:13]
+; SI-SDAG-NEXT: s_mov_b32 s16, 0x623fde64
+; SI-SDAG-NEXT: s_mov_b32 s17, 0x3ec71dee
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[16:17]
+; SI-SDAG-NEXT: s_mov_b32 s20, 0x7c89e6b0
+; SI-SDAG-NEXT: s_mov_b32 s21, 0x3efa0199
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[20:21]
+; SI-SDAG-NEXT: s_mov_b32 s22, 0x14761f6e
+; SI-SDAG-NEXT: s_mov_b32 s23, 0x3f2a01a0
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[22:23]
+; SI-SDAG-NEXT: s_mov_b32 s24, 0x1852b7b0
+; SI-SDAG-NEXT: s_mov_b32 s25, 0x3f56c16c
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[24:25]
+; SI-SDAG-NEXT: s_mov_b32 s26, 0x11122322
+; SI-SDAG-NEXT: s_mov_b32 s27, 0x3f811111
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[26:27]
+; SI-SDAG-NEXT: s_mov_b32 s28, 0x555502a1
+; SI-SDAG-NEXT: s_mov_b32 s29, 0x3fa55555
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[28:29]
+; SI-SDAG-NEXT: s_mov_b32 s40, 0x55555511
+; SI-SDAG-NEXT: s_mov_b32 s41, 0x3fc55555
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[40:41]
+; SI-SDAG-NEXT: s_mov_b32 s42, 11
+; SI-SDAG-NEXT: s_mov_b32 s43, 0x3fe00000
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], s[42:43]
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v9, v[17:18]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[15:16], v[21:22], 1.0
+; SI-SDAG-NEXT: s_mov_b32 s46, 0
+; SI-SDAG-NEXT: v_fma_f64 v[15:16], v[15:16], v[21:22], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], s[18:19], v[12:13]
+; SI-SDAG-NEXT: s_mov_b32 s47, 0x40900000
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[16:17]
+; SI-SDAG-NEXT: s_mov_b32 s44, 0
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[20:21]
+; SI-SDAG-NEXT: s_mov_b32 s45, 0xc090cc00
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[22:23]
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[46:47], v[2:3]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[24:25]
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[2:3]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[26:27]
+; SI-SDAG-NEXT: v_cmp_gt_f64_e64 s[6:7], |v[4:5]|, s[10:11]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[28:29]
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[8:9], s[44:45], v[0:1]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[40:41]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], s[42:43]
+; SI-SDAG-NEXT: v_fma_f64 v[21:22], v[19:20], v[21:22], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[19:20], v[19:20], v[21:22], 1.0
+; SI-SDAG-NEXT: v_mov_b32_e32 v22, 0x7ff00000
+; SI-SDAG-NEXT: v_ldexp_f64 v[17:18], v[19:20], v9
+; SI-SDAG-NEXT: v_bfi_b32 v9, s56, v14, v5
+; SI-SDAG-NEXT: v_add_f64 v[2:3], v[4:5], v[8:9]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v23, v22, v18, vcc
+; SI-SDAG-NEXT: v_add_f64 v[2:3], v[2:3], -v[8:9]
+; SI-SDAG-NEXT: v_bfi_b32 v9, s56, v14, v7
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v19, v3, v5, s[6:7]
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v18, v2, v4, s[6:7]
+; SI-SDAG-NEXT: v_add_f64 v[2:3], v[4:5], -v[18:19]
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[6:7], s[46:47], v[0:1]
+; SI-SDAG-NEXT: v_mul_f64 v[20:21], v[2:3], s[12:13]
+; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
+; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[2:3], s[14:15], v[20:21]
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v2, v[10:11]
+; SI-SDAG-NEXT: v_add_f64 v[10:11], v[6:7], v[8:9]
+; SI-SDAG-NEXT: v_add_f64 v[8:9], v[10:11], -v[8:9]
+; SI-SDAG-NEXT: v_ldexp_f64 v[2:3], v[15:16], v2
+; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v16, v[18:19]
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, v22, v3, s[6:7]
+; SI-SDAG-NEXT: s_and_b64 s[6:7], s[8:9], s[6:7]
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v0, 0, v2, s[6:7]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v2, 0, v17, vcc
+; SI-SDAG-NEXT: v_cmp_gt_f64_e64 vcc, |v[6:7]|, s[10:11]
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v1, 0, v3, s[8:9]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v9, v7, vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v8, v8, v6, vcc
+; SI-SDAG-NEXT: v_add_f64 v[10:11], v[6:7], -v[8:9]
; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[14:15], v[18:19], 1.0
-; SI-SDAG-NEXT: s_mov_b32 s58, 0
-; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[14:15], v[18:19], 1.0
-; SI-SDAG-NEXT: s_mov_b32 s59, 0x40900000
-; SI-SDAG-NEXT: s_mov_b32 s60, 0
-; SI-SDAG-NEXT: s_mov_b32 s61, 0xc090cc00
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[58:59], v[0:1]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[60:61], v[0:1]
-; SI-SDAG-NEXT: v_bfi_b32 v11, s62, v17, v3
-; SI-SDAG-NEXT: v_ldexp_f64 v[8:9], v[14:15], v8
-; SI-SDAG-NEXT: v_add_f64 v[14:15], v[2:3], v[10:11]
-; SI-SDAG-NEXT: s_and_b64 s[6:7], s[4:5], vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v0, 0, v8, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[14:15], v[14:15], -v[10:11]
-; SI-SDAG-NEXT: v_cmp_gt_f64_e64 s[6:7], |v[2:3]|, s[14:15]
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[16:17], s[58:59], v[2:3]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v15, v15, v3, s[6:7]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v14, v14, v2, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[18:19], v[2:3], -v[14:15]
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v1, v[14:15]
-; SI-SDAG-NEXT: v_mul_f64 v[20:21], v[18:19], s[18:19]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[10:11], s[60:61], v[2:3]
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[18:19], s[20:21], v[20:21]
-; SI-SDAG-NEXT: s_and_b64 s[6:7], s[10:11], s[16:17]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], s[22:23], v[12:13]
-; SI-SDAG-NEXT: v_bfi_b32 v11, s62, v17, v5
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[24:25]
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[8:9], s[58:59], v[4:5]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[26:27]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[28:29]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[40:41]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[42:43]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[44:45]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[46:47]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[56:57]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[18:19], v[20:21], 1.0
-; SI-SDAG-NEXT: v_ldexp_f64 v[15:16], v[18:19], v1
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v2, 0, v15, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[14:15], v[4:5], v[10:11]
-; SI-SDAG-NEXT: v_cmp_gt_f64_e64 s[6:7], |v[4:5]|, s[14:15]
-; SI-SDAG-NEXT: v_add_f64 v[14:15], v[14:15], -v[10:11]
-; SI-SDAG-NEXT: v_bfi_b32 v11, s62, v17, v7
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v15, v15, v5, s[6:7]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v14, v14, v4, s[6:7]
-; SI-SDAG-NEXT: v_add_f64 v[18:19], v[4:5], -v[14:15]
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v1, v[14:15]
-; SI-SDAG-NEXT: v_mul_f64 v[20:21], v[18:19], s[18:19]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[6:7], s[60:61], v[4:5]
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[18:19], s[20:21], v[20:21]
-; SI-SDAG-NEXT: s_and_b64 s[12:13], s[6:7], s[8:9]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], s[22:23], v[12:13]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[24:25]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[26:27]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[28:29]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[40:41]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[42:43]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[44:45]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[46:47]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], s[56:57]
-; SI-SDAG-NEXT: v_fma_f64 v[20:21], v[18:19], v[20:21], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[18:19], v[18:19], v[20:21], 1.0
-; SI-SDAG-NEXT: v_ldexp_f64 v[14:15], v[18:19], v1
-; SI-SDAG-NEXT: v_add_f64 v[17:18], v[6:7], v[10:11]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v4, 0, v14, s[12:13]
-; SI-SDAG-NEXT: v_add_f64 v[10:11], v[17:18], -v[10:11]
-; SI-SDAG-NEXT: v_cmp_gt_f64_e64 s[12:13], |v[6:7]|, s[14:15]
-; SI-SDAG-NEXT: v_cmp_nlt_f64_e64 s[14:15], s[58:59], v[6:7]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v11, v11, v7, s[12:13]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v10, v10, v6, s[12:13]
-; SI-SDAG-NEXT: v_add_f64 v[17:18], v[6:7], -v[10:11]
-; SI-SDAG-NEXT: v_cvt_i32_f64_e32 v1, v[10:11]
-; SI-SDAG-NEXT: v_mul_f64 v[19:20], v[17:18], s[18:19]
-; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[12:13], s[60:61], v[6:7]
-; SI-SDAG-NEXT: v_fma_f64 v[17:18], v[17:18], s[20:21], v[19:20]
-; SI-SDAG-NEXT: v_mov_b32_e32 v7, 0x7ff00000
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], s[22:23], v[12:13]
-; SI-SDAG-NEXT: s_and_b64 s[18:19], s[12:13], s[14:15]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[24:25]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, v7, v16, s[16:17]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[26:27]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v5, v7, v15, s[8:9]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[28:29]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, 0, v3, s[10:11]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[40:41]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v5, 0, v5, s[6:7]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[42:43]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[44:45]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[46:47]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], s[56:57]
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], 1.0
-; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[17:18], v[12:13], 1.0
-; SI-SDAG-NEXT: v_ldexp_f64 v[10:11], v[12:13], v1
-; SI-SDAG-NEXT: v_cndmask_b32_e32 v1, v7, v9, vcc
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v7, v7, v11, s[14:15]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v6, 0, v10, s[18:19]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v1, 0, v1, s[4:5]
-; SI-SDAG-NEXT: v_cndmask_b32_e64 v7, 0, v7, s[12:13]
+; SI-SDAG-NEXT: v_mul_f64 v[14:15], v[10:11], s[12:13]
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v3, 0, v23, s[4:5]
+; SI-SDAG-NEXT: v_fma_f64 v[10:11], v[10:11], s[14:15], v[14:15]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], s[18:19], v[12:13]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], s[18:19], v[12:13]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[16:17]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[16:17]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[20:21]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[20:21]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[22:23]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[22:23]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[24:25]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[24:25]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[26:27]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[26:27]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[28:29]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[28:29]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[40:41]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[40:41]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], s[42:43]
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], s[42:43]
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[12:13], v[10:11], v[12:13], 1.0
+; SI-SDAG-NEXT: v_fma_f64 v[14:15], v[20:21], v[14:15], 1.0
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[46:47], v[4:5]
+; SI-SDAG-NEXT: v_ldexp_f64 v[14:15], v[14:15], v16
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[4:5]
+; SI-SDAG-NEXT: v_fma_f64 v[10:11], v[10:11], v[12:13], 1.0
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v22, v15, vcc
+; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v5, 0, v9, s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v4, 0, v14, vcc
+; SI-SDAG-NEXT: v_ldexp_f64 v[8:9], v[10:11], v8
+; SI-SDAG-NEXT: v_cmp_nlt_f64_e32 vcc, s[46:47], v[6:7]
+; SI-SDAG-NEXT: v_cmp_ngt_f64_e64 s[4:5], s[44:45], v[6:7]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v9, v22, v9, vcc
+; SI-SDAG-NEXT: s_and_b64 vcc, s[4:5], vcc
+; SI-SDAG-NEXT: v_cndmask_b32_e64 v7, 0, v9, s[4:5]
+; SI-SDAG-NEXT: v_cndmask_b32_e32 v6, 0, v8, vcc
; SI-SDAG-NEXT: s_setpc_b64 s[30:31]
;
; SI-GISEL-LABEL: v_exp2_v4f64:
; SI-GISEL: ; %bb.0:
; SI-GISEL-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-GISEL-NEXT: v_and_b32_e32 v8, 0x80000000, v1
-; SI-GISEL-NEXT: v_mov_b32_e32 v10, 0
-; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v8
-; SI-GISEL-NEXT: v_add_f64 v[8:9], v[0:1], v[10:11]
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, -1
-; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x432fffff
-; SI-GISEL-NEXT: v_add_f64 v[8:9], v[8:9], -v[10:11]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[0:1]|, v[18:19]
-; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v3
+; SI-GISEL-NEXT: v_mov_b32_e32 v16, 0
+; SI-GISEL-NEXT: v_or_b32_e32 v17, 0x43300000, v8
+; SI-GISEL-NEXT: v_add_f64 v[8:9], v[0:1], v[16:17]
+; SI-GISEL-NEXT: v_mov_b32_e32 v14, -1
+; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x432fffff
+; SI-GISEL-NEXT: v_add_f64 v[8:9], v[8:9], -v[16:17]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[0:1]|, v[14:15]
+; SI-GISEL-NEXT: v_and_b32_e32 v17, 0x80000000, v3
; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, v8, v0, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v9, v1, vcc
-; SI-GISEL-NEXT: v_add_f64 v[16:17], v[0:1], -v[8:9]
-; SI-GISEL-NEXT: v_mov_b32_e32 v14, 0x3b39803f
-; SI-GISEL-NEXT: v_mov_b32_e32 v15, 0x3c7abc9e
-; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
-; SI-GISEL-NEXT: v_mul_f64 v[20:21], v[16:17], v[14:15]
-; SI-GISEL-NEXT: v_add_f64 v[22:23], v[2:3], v[10:11]
-; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0xfefa39ef
-; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x3fe62e42
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[16:17], v[12:13], v[20:21]
-; SI-GISEL-NEXT: v_add_f64 v[16:17], v[22:23], -v[10:11]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[2:3]|, v[18:19]
-; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v5
-; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v20, v16, v2, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v21, v17, v3, vcc
-; SI-GISEL-NEXT: v_add_f64 v[16:17], v[4:5], v[10:11]
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[4:5]|, v[18:19]
-; SI-GISEL-NEXT: v_add_f64 v[16:17], v[16:17], -v[10:11]
-; SI-GISEL-NEXT: v_add_f64 v[22:23], v[2:3], -v[20:21]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, v16, v4, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v17, v5, vcc
-; SI-GISEL-NEXT: v_add_f64 v[26:27], v[4:5], -v[16:17]
-; SI-GISEL-NEXT: v_mul_f64 v[28:29], v[22:23], v[14:15]
-; SI-GISEL-NEXT: v_mul_f64 v[30:31], v[26:27], v[14:15]
-; SI-GISEL-NEXT: v_and_b32_e32 v11, 0x80000000, v7
-; SI-GISEL-NEXT: v_or_b32_e32 v11, 0x43300000, v11
-; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[22:23], v[12:13], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[22:23], v[26:27], v[12:13], v[30:31]
-; SI-GISEL-NEXT: v_add_f64 v[26:27], v[6:7], v[10:11]
+; SI-GISEL-NEXT: v_or_b32_e32 v17, 0x43300000, v17
+; SI-GISEL-NEXT: v_add_f64 v[10:11], v[0:1], -v[8:9]
+; SI-GISEL-NEXT: v_add_f64 v[20:21], v[2:3], v[16:17]
+; SI-GISEL-NEXT: v_mov_b32_e32 v12, 0x3b39803f
+; SI-GISEL-NEXT: v_mov_b32_e32 v13, 0x3c7abc9e
+; SI-GISEL-NEXT: v_mul_f64 v[18:19], v[10:11], v[12:13]
+; SI-GISEL-NEXT: v_add_f64 v[20:21], v[20:21], -v[16:17]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[2:3]|, v[14:15]
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0xfefa39ef
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x3fe62e42
+; SI-GISEL-NEXT: v_and_b32_e32 v17, 0x80000000, v5
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[10:11], v[22:23], v[18:19]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, v20, v2, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v21, v3, vcc
+; SI-GISEL-NEXT: v_or_b32_e32 v17, 0x43300000, v17
+; SI-GISEL-NEXT: v_add_f64 v[18:19], v[2:3], -v[10:11]
+; SI-GISEL-NEXT: v_add_f64 v[20:21], v[4:5], v[16:17]
+; SI-GISEL-NEXT: v_mul_f64 v[26:27], v[18:19], v[12:13]
+; SI-GISEL-NEXT: v_add_f64 v[20:21], v[20:21], -v[16:17]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[4:5]|, v[14:15]
+; SI-GISEL-NEXT: v_and_b32_e32 v17, 0x80000000, v7
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[18:19], v[22:23], v[26:27]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v18, v20, v4, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v19, v21, v5, vcc
+; SI-GISEL-NEXT: v_or_b32_e32 v17, 0x43300000, v17
+; SI-GISEL-NEXT: v_add_f64 v[20:21], v[6:7], v[16:17]
+; SI-GISEL-NEXT: v_add_f64 v[28:29], v[4:5], -v[18:19]
+; SI-GISEL-NEXT: v_add_f64 v[16:17], v[20:21], -v[16:17]
+; SI-GISEL-NEXT: v_mul_f64 v[20:21], v[28:29], v[12:13]
; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0x6a5dcb37
; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x3e5ade15
; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0xfca7ab0c
; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x3e928af3
-; SI-GISEL-NEXT: v_add_f64 v[10:11], v[26:27], -v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[24:25], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[28:29], v[22:23], v[20:21]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[24:25], v[30:31], v[32:33]
; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0x623fde64
; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0x3ec71dee
-; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[6:7]|, v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[24:25], v[26:27], v[34:35]
-; SI-GISEL-NEXT: v_mov_b32_e32 v26, 0x7c89e6b0
-; SI-GISEL-NEXT: v_mov_b32_e32 v27, 0x3efa0199
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[24:25], v[18:19], v[26:27]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v10, v10, v6, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v11, v11, v7, vcc
-; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x14761f6e
-; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3f2a01a0
-; SI-GISEL-NEXT: v_add_f64 v[38:39], v[6:7], -v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[24:25], v[18:19], v[36:37]
+; SI-GISEL-NEXT: v_cmp_gt_f64_e64 vcc, |v[6:7]|, v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[24:25], v[28:29], v[34:35]
+; SI-GISEL-NEXT: v_mov_b32_e32 v36, 0x7c89e6b0
+; SI-GISEL-NEXT: v_mov_b32_e32 v37, 0x3efa0199
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v14, v16, v6, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v15, v17, v7, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[28:29], v[36:37]
+; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0x14761f6e
+; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0x3f2a01a0
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], v[28:29]
+; SI-GISEL-NEXT: v_add_f64 v[38:39], v[6:7], -v[14:15]
; SI-GISEL-NEXT: v_mov_b32_e32 v48, 0x1852b7b0
; SI-GISEL-NEXT: v_mov_b32_e32 v49, 0x3f56c16c
-; SI-GISEL-NEXT: v_mul_f64 v[14:15], v[38:39], v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[24:25], v[18:19], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], v[48:49]
+; SI-GISEL-NEXT: v_mul_f64 v[12:13], v[38:39], v[12:13]
; SI-GISEL-NEXT: v_mov_b32_e32 v50, 0x11122322
; SI-GISEL-NEXT: v_mov_b32_e32 v51, 0x3f811111
-; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[38:39], v[12:13], v[14:15]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[24:25], v[18:19], v[50:51]
-; SI-GISEL-NEXT: v_mov_b32_e32 v18, 0x555502a1
-; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x3fa55555
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[24:25], v[14:15], v[18:19]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[12:13], v[38:39], v[22:23], v[12:13]
+; SI-GISEL-NEXT: v_mov_b32_e32 v22, 0x555502a1
+; SI-GISEL-NEXT: v_mov_b32_e32 v23, 0x3fa55555
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], v[22:23]
; SI-GISEL-NEXT: v_mov_b32_e32 v38, 0x55555511
; SI-GISEL-NEXT: v_mov_b32_e32 v39, 0x3fc55555
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[24:25], v[14:15], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], v[38:39]
; SI-GISEL-NEXT: v_mov_b32_e32 v52, 11
; SI-GISEL-NEXT: v_mov_b32_e32 v53, 0x3fe00000
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[24:25], v[14:15], v[52:53]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[8:9]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[24:25], v[14:15], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v10, v[10:11]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[24:25], v[14:15], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[22:23], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[14:15], v8
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], v[52:53]
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[14:15], v[28:29], v[14:15], 1.0
-; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[12:13], v[30:31], v[32:33]
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v32, v[20:21]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[24:25], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[28:29], v[34:35]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[26:27]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[36:37]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[48:49]
-; SI-GISEL-NEXT: v_mov_b32_e32 v30, 0
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[50:51]
-; SI-GISEL-NEXT: v_mov_b32_e32 v31, 0x40900000
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[18:19]
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[30:31]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[38:39]
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[48:49]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], v[52:53]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v33, 0, v8, vcc
-; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[12:13], v[24:25], v[50:51]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], 1.0
-; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v8, v[16:17]
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[12:13], v[24:25], v[18:19]
-; SI-GISEL-NEXT: v_fma_f64 v[20:21], v[22:23], v[20:21], 1.0
-; SI-GISEL-NEXT: v_mov_b32_e32 v16, 0x7ff00000
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[12:13], v[18:19], v[38:39]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v16, v9, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[20:21], v8
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[30:31]
-; SI-GISEL-NEXT: v_mov_b32_e32 v28, 0
-; SI-GISEL-NEXT: v_mov_b32_e32 v29, 0xc090cc00
-; SI-GISEL-NEXT: v_fma_f64 v[18:19], v[12:13], v[18:19], v[52:53]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], v[52:53]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v18, v[18:19]
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], 1.0
+; SI-GISEL-NEXT: v_mov_b32_e32 v19, 0x7ff00000
+; SI-GISEL-NEXT: v_fma_f64 v[16:17], v[24:25], v[16:17], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v14, v[14:15]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[38:39]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], v[52:53]
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[24:25], v[26:27], v[24:25], 1.0
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[12:13], v[30:31], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[26:27], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[12:13], v[30:31], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[26:27], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[30:31], v[12:13], v[30:31], v[36:37]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[26:27], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[12:13], v[30:31], v[28:29]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[26:27], v[48:49]
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v30, v[8:9]
+; SI-GISEL-NEXT: v_fma_f64 v[26:27], v[20:21], v[26:27], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[12:13], v[28:29], v[48:49]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[26:27], v[22:23]
+; SI-GISEL-NEXT: v_fma_f64 v[28:29], v[12:13], v[28:29], v[50:51]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], v[38:39]
+; SI-GISEL-NEXT: v_mov_b32_e32 v32, 0
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], v[52:53]
+; SI-GISEL-NEXT: v_mov_b32_e32 v33, 0x40900000
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cvt_i32_f64_e32 v31, v[10:11]
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[28:29], v[22:23]
+; SI-GISEL-NEXT: v_ldexp_f64 v[16:17], v[16:17], v30
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[0:1], v[32:33]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[20:21], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v16, 0, v16, vcc
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[38:39]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v17, v19, v17, vcc
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v18
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[4:5], v[32:33]
+; SI-GISEL-NEXT: v_mov_b32_e32 v34, 0
+; SI-GISEL-NEXT: v_mov_b32_e32 v35, 0xc090cc00
+; SI-GISEL-NEXT: v_fma_f64 v[10:11], v[12:13], v[10:11], v[52:53]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v20, 0, v8, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v21, v16, v9, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[28:29]
-; SI-GISEL-NEXT: v_ldexp_f64 v[14:15], v[14:15], v32
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e64 s[4:5], v[2:3], v[30:31]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v33, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v21, v19, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[0:1], v[34:35]
+; SI-GISEL-NEXT: v_ldexp_f64 v[22:23], v[24:25], v31
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e64 s[4:5], v[2:3], v[32:33]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v0, 0, v16, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v1, 0, v17, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[28:29]
-; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[12:13], v[18:19], 1.0
-; SI-GISEL-NEXT: v_cndmask_b32_e64 v14, 0, v14, s[4:5]
-; SI-GISEL-NEXT: v_cndmask_b32_e64 v15, v16, v15, s[4:5]
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v14, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v15, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[28:29]
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[2:3], v[34:35]
+; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[12:13], v[10:11], 1.0
+; SI-GISEL-NEXT: v_cndmask_b32_e64 v22, 0, v22, s[4:5]
+; SI-GISEL-NEXT: v_cndmask_b32_e64 v18, v19, v23, s[4:5]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v2, 0, v22, vcc
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v3, 0, v18, vcc
; SI-GISEL-NEXT: v_fma_f64 v[8:9], v[12:13], v[8:9], 1.0
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[4:5], v[34:35]
+; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v14
; SI-GISEL-NEXT: v_cndmask_b32_e32 v4, 0, v20, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v5, 0, v21, vcc
-; SI-GISEL-NEXT: v_ldexp_f64 v[8:9], v[8:9], v10
-; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[6:7], v[30:31]
+; SI-GISEL-NEXT: v_cmp_ngt_f64_e32 vcc, v[6:7], v[32:33]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v8, 0, v8, vcc
-; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
-; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[6:7], v[28:29]
+; SI-GISEL-NEXT: v_cndmask_b32_e32 v9, v19, v9, vcc
+; SI-GISEL-NEXT: v_cmp_nlt_f64_e32 vcc, v[6:7], v[34:35]
; SI-GISEL-NEXT: v_cndmask_b32_e32 v6, 0, v8, vcc
; SI-GISEL-NEXT: v_cndmask_b32_e32 v7, 0, v9, vcc
; SI-GISEL-NEXT: s_setpc_b64 s[30:31]
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.maximum.f16.ll b/llvm/test/CodeGen/AMDGPU/llvm.maximum.f16.ll
index fe452122d788d..75ca48958400e 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.maximum.f16.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.maximum.f16.ll
@@ -2604,87 +2604,87 @@ define <16 x half> @v_maximum_v16f16(<16 x half> %src0, <16 x half> %src1) {
; GFX8-LABEL: v_maximum_v16f16:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v14
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v6
-; GFX8-NEXT: v_max_f16_e32 v16, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v18, v17
+; GFX8-NEXT: v_lshrrev_b32_e32 v16, 16, v14
+; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v6
+; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v17, v16
+; GFX8-NEXT: v_max_f16_e32 v16, v17, v16
; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v13
; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v5
-; GFX8-NEXT: v_max_f16_e32 v20, v18, v17
; GFX8-NEXT: v_cmp_o_f16_e64 s[4:5], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v12
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v4
-; GFX8-NEXT: v_max_f16_e32 v21, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[6:7], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v11
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v3
-; GFX8-NEXT: v_max_f16_e32 v22, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[8:9], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v10
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v2
-; GFX8-NEXT: v_max_f16_e32 v23, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[10:11], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v9
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v1
-; GFX8-NEXT: v_mov_b32_e32 v19, 0x7e00
-; GFX8-NEXT: v_max_f16_e32 v24, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[12:13], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v8
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v0
-; GFX8-NEXT: v_max_f16_e32 v25, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[14:15], v18, v17
-; GFX8-NEXT: v_max_f16_e32 v17, v6, v14
+; GFX8-NEXT: v_max_f16_e32 v17, v18, v17
+; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v12
+; GFX8-NEXT: v_lshrrev_b32_e32 v19, 16, v4
+; GFX8-NEXT: v_cmp_o_f16_e64 s[6:7], v19, v18
+; GFX8-NEXT: v_max_f16_e32 v18, v19, v18
+; GFX8-NEXT: v_lshrrev_b32_e32 v19, 16, v11
+; GFX8-NEXT: v_lshrrev_b32_e32 v20, 16, v3
+; GFX8-NEXT: v_cmp_o_f16_e64 s[8:9], v20, v19
+; GFX8-NEXT: v_max_f16_e32 v19, v20, v19
+; GFX8-NEXT: v_lshrrev_b32_e32 v20, 16, v10
+; GFX8-NEXT: v_lshrrev_b32_e32 v21, 16, v2
+; GFX8-NEXT: v_cmp_o_f16_e64 s[10:11], v21, v20
+; GFX8-NEXT: v_max_f16_e32 v20, v21, v20
+; GFX8-NEXT: v_lshrrev_b32_e32 v21, 16, v9
+; GFX8-NEXT: v_lshrrev_b32_e32 v22, 16, v1
+; GFX8-NEXT: v_cmp_o_f16_e64 s[12:13], v22, v21
+; GFX8-NEXT: v_max_f16_e32 v21, v22, v21
+; GFX8-NEXT: v_lshrrev_b32_e32 v22, 16, v8
+; GFX8-NEXT: v_lshrrev_b32_e32 v23, 16, v0
+; GFX8-NEXT: v_cmp_o_f16_e64 s[14:15], v23, v22
+; GFX8-NEXT: v_max_f16_e32 v22, v23, v22
+; GFX8-NEXT: v_cmp_o_f16_e64 s[22:23], v3, v11
+; GFX8-NEXT: v_max_f16_e32 v3, v3, v11
+; GFX8-NEXT: v_cmp_o_f16_e64 s[24:25], v2, v10
+; GFX8-NEXT: v_max_f16_e32 v2, v2, v10
+; GFX8-NEXT: v_cmp_o_f16_e64 s[26:27], v7, v15
+; GFX8-NEXT: v_max_f16_e32 v10, v7, v15
+; GFX8-NEXT: v_lshrrev_b32_e32 v11, 16, v15
+; GFX8-NEXT: v_lshrrev_b32_e32 v7, 16, v7
+; GFX8-NEXT: v_cmp_o_f16_e64 s[42:43], v0, v8
+; GFX8-NEXT: v_max_f16_e32 v0, v0, v8
+; GFX8-NEXT: v_mov_b32_e32 v8, 0x7e00
; GFX8-NEXT: v_cmp_o_f16_e64 s[16:17], v6, v14
-; GFX8-NEXT: v_max_f16_e32 v6, v5, v13
+; GFX8-NEXT: v_max_f16_e32 v6, v6, v14
; GFX8-NEXT: v_cmp_o_f16_e64 s[18:19], v5, v13
-; GFX8-NEXT: v_max_f16_e32 v5, v4, v12
+; GFX8-NEXT: v_max_f16_e32 v5, v5, v13
; GFX8-NEXT: v_cmp_o_f16_e64 s[20:21], v4, v12
-; GFX8-NEXT: v_max_f16_e32 v4, v3, v11
-; GFX8-NEXT: v_cmp_o_f16_e64 s[22:23], v3, v11
-; GFX8-NEXT: v_max_f16_e32 v3, v2, v10
-; GFX8-NEXT: v_max_f16_e32 v11, v7, v15
-; GFX8-NEXT: v_cmp_o_f16_e64 s[24:25], v7, v15
-; GFX8-NEXT: v_lshrrev_b32_e32 v12, 16, v15
-; GFX8-NEXT: v_lshrrev_b32_e32 v7, 16, v7
-; GFX8-NEXT: v_cndmask_b32_e32 v14, v19, v16, vcc
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v2, v10
-; GFX8-NEXT: v_max_f16_e32 v13, v7, v12
-; GFX8-NEXT: v_cmp_o_f16_e64 s[26:27], v7, v12
-; GFX8-NEXT: v_max_f16_e32 v7, v1, v9
-; GFX8-NEXT: v_cndmask_b32_e32 v2, v19, v3, vcc
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v1, v9
-; GFX8-NEXT: v_max_f16_e32 v12, v0, v8
-; GFX8-NEXT: v_cndmask_b32_e64 v18, v19, v22, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v22, v19, v25, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v19, v7, vcc
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v0, v8
-; GFX8-NEXT: v_cndmask_b32_e64 v16, v19, v21, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v21, v19, v24, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e32 v0, v19, v12, vcc
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v22
-; GFX8-NEXT: v_cndmask_b32_e64 v15, v19, v20, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v20, v19, v23, s[10:11]
-; GFX8-NEXT: v_or_b32_sdwa v0, v0, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v21
-; GFX8-NEXT: v_or_b32_sdwa v1, v1, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v20
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v19, v4, s[22:23]
-; GFX8-NEXT: v_or_b32_sdwa v2, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v18
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v19, v5, s[20:21]
-; GFX8-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v4, 16, v16
-; GFX8-NEXT: v_cndmask_b32_e64 v13, v19, v13, s[26:27]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v19, v6, s[18:19]
-; GFX8-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v5, 16, v15
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v19, v11, s[24:25]
-; GFX8-NEXT: v_cndmask_b32_e64 v17, v19, v17, s[16:17]
-; GFX8-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v6, 16, v14
-; GFX8-NEXT: v_lshlrev_b32_e32 v7, 16, v13
-; GFX8-NEXT: v_or_b32_sdwa v6, v17, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_or_b32_sdwa v7, v11, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_max_f16_e32 v4, v4, v12
+; GFX8-NEXT: v_cmp_o_f16_e64 s[28:29], v7, v11
+; GFX8-NEXT: v_max_f16_e32 v7, v7, v11
+; GFX8-NEXT: v_cmp_o_f16_e64 s[40:41], v1, v9
+; GFX8-NEXT: v_max_f16_e32 v1, v1, v9
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v8, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v16, v8, v22, s[14:15]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v8, v7, s[28:29]
+; GFX8-NEXT: v_cndmask_b32_e64 v11, v8, v17, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v12, v8, v18, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v13, v8, v19, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v14, v8, v20, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v15, v8, v21, s[12:13]
+; GFX8-NEXT: v_cndmask_b32_e64 v10, v8, v10, s[26:27]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v8, v6, s[16:17]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v8, v5, s[18:19]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v8, v4, s[20:21]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v8, v3, s[22:23]
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v8, v2, s[24:25]
+; GFX8-NEXT: v_cndmask_b32_e64 v1, v8, v1, s[40:41]
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v8, v0, s[42:43]
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v16
+; GFX8-NEXT: v_or_b32_sdwa v0, v0, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v15
+; GFX8-NEXT: v_or_b32_sdwa v1, v1, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v14
+; GFX8-NEXT: v_or_b32_sdwa v2, v2, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v13
+; GFX8-NEXT: v_or_b32_sdwa v3, v3, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v12
+; GFX8-NEXT: v_or_b32_sdwa v4, v4, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v11
+; GFX8-NEXT: v_or_b32_sdwa v5, v5, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v9
+; GFX8-NEXT: v_lshlrev_b32_e32 v7, 16, v7
+; GFX8-NEXT: v_or_b32_sdwa v6, v6, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_or_b32_sdwa v7, v10, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX900-LABEL: v_maximum_v16f16:
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.maximum.f32.ll b/llvm/test/CodeGen/AMDGPU/llvm.maximum.f32.ll
index 6d757b07bca1a..b536af7bd1ddf 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.maximum.f32.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.maximum.f32.ll
@@ -1836,169 +1836,169 @@ define <16 x float> @v_maximum_v16f32(<16 x float> %src0, <16 x float> %src1) {
; GFX7-LABEL: v_maximum_v16f32:
; GFX7: ; %bb.0:
; GFX7-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
; GFX7-NEXT: v_max_f32_e32 v0, v0, v16
-; GFX7-NEXT: buffer_load_dword v16, off, s[0:3], s32
-; GFX7-NEXT: v_cmp_o_f32_e64 s[4:5], v1, v17
+; GFX7-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX7-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
; GFX7-NEXT: v_max_f32_e32 v1, v1, v17
-; GFX7-NEXT: v_cmp_o_f32_e64 s[6:7], v2, v18
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v16, v1, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v2, v18
; GFX7-NEXT: v_max_f32_e32 v2, v2, v18
-; GFX7-NEXT: v_mov_b32_e32 v17, 0x7fc00000
-; GFX7-NEXT: v_max_f32_e32 v18, v13, v29
-; GFX7-NEXT: v_cmp_o_f32_e64 s[28:29], v13, v29
-; GFX7-NEXT: v_cmp_o_f32_e64 s[8:9], v3, v19
+; GFX7-NEXT: v_cndmask_b32_e32 v2, v16, v2, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v3, v19
; GFX7-NEXT: v_max_f32_e32 v3, v3, v19
-; GFX7-NEXT: v_cmp_o_f32_e64 s[10:11], v4, v20
+; GFX7-NEXT: v_cndmask_b32_e32 v3, v16, v3, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v4, v20
; GFX7-NEXT: v_max_f32_e32 v4, v4, v20
-; GFX7-NEXT: v_cmp_o_f32_e64 s[12:13], v5, v21
+; GFX7-NEXT: v_cndmask_b32_e32 v4, v16, v4, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v5, v21
; GFX7-NEXT: v_max_f32_e32 v5, v5, v21
-; GFX7-NEXT: v_cmp_o_f32_e64 s[14:15], v6, v22
+; GFX7-NEXT: v_cndmask_b32_e32 v5, v16, v5, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v6, v22
; GFX7-NEXT: v_max_f32_e32 v6, v6, v22
-; GFX7-NEXT: v_cmp_o_f32_e64 s[16:17], v7, v23
+; GFX7-NEXT: v_cndmask_b32_e32 v6, v16, v6, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v7, v23
; GFX7-NEXT: v_max_f32_e32 v7, v7, v23
-; GFX7-NEXT: v_cmp_o_f32_e64 s[18:19], v8, v24
+; GFX7-NEXT: v_cndmask_b32_e32 v7, v16, v7, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
; GFX7-NEXT: v_max_f32_e32 v8, v8, v24
-; GFX7-NEXT: v_cmp_o_f32_e64 s[20:21], v9, v25
+; GFX7-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
; GFX7-NEXT: v_max_f32_e32 v9, v9, v25
-; GFX7-NEXT: v_cmp_o_f32_e64 s[22:23], v10, v26
+; GFX7-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
; GFX7-NEXT: v_max_f32_e32 v10, v10, v26
-; GFX7-NEXT: v_cmp_o_f32_e64 s[24:25], v11, v27
+; GFX7-NEXT: v_cndmask_b32_e32 v10, v16, v10, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
; GFX7-NEXT: v_max_f32_e32 v11, v11, v27
-; GFX7-NEXT: v_cmp_o_f32_e64 s[26:27], v12, v28
+; GFX7-NEXT: v_cndmask_b32_e32 v11, v16, v11, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
; GFX7-NEXT: v_max_f32_e32 v12, v12, v28
-; GFX7-NEXT: v_max_f32_e32 v19, v14, v30
-; GFX7-NEXT: v_cmp_o_f32_e64 s[40:41], v14, v30
-; GFX7-NEXT: v_cndmask_b32_e32 v0, v17, v0, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v13, v17, v18, s[28:29]
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v17, v1, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[14:15]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[16:17]
-; GFX7-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[18:19]
-; GFX7-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[20:21]
-; GFX7-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[22:23]
-; GFX7-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[24:25]
-; GFX7-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[26:27]
-; GFX7-NEXT: v_cndmask_b32_e64 v14, v17, v19, s[40:41]
+; GFX7-NEXT: v_cndmask_b32_e32 v12, v16, v12, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
+; GFX7-NEXT: v_max_f32_e32 v13, v13, v29
+; GFX7-NEXT: v_cndmask_b32_e32 v13, v16, v13, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX7-NEXT: v_max_f32_e32 v14, v14, v30
+; GFX7-NEXT: v_cndmask_b32_e32 v14, v16, v14, vcc
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: v_max_f32_e32 v18, v15, v16
-; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v15, v16
-; GFX7-NEXT: v_cndmask_b32_e32 v15, v17, v18, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX7-NEXT: v_max_f32_e32 v15, v15, v31
+; GFX7-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: v_maximum_v16f32:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX8-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
; GFX8-NEXT: v_max_f32_e32 v0, v0, v16
-; GFX8-NEXT: buffer_load_dword v16, off, s[0:3], s32
-; GFX8-NEXT: v_cmp_o_f32_e64 s[4:5], v1, v17
+; GFX8-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX8-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
; GFX8-NEXT: v_max_f32_e32 v1, v1, v17
-; GFX8-NEXT: v_cmp_o_f32_e64 s[6:7], v2, v18
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v16, v1, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v2, v18
; GFX8-NEXT: v_max_f32_e32 v2, v2, v18
-; GFX8-NEXT: v_mov_b32_e32 v17, 0x7fc00000
-; GFX8-NEXT: v_max_f32_e32 v18, v13, v29
-; GFX8-NEXT: v_cmp_o_f32_e64 s[28:29], v13, v29
-; GFX8-NEXT: v_cmp_o_f32_e64 s[8:9], v3, v19
+; GFX8-NEXT: v_cndmask_b32_e32 v2, v16, v2, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v3, v19
; GFX8-NEXT: v_max_f32_e32 v3, v3, v19
-; GFX8-NEXT: v_cmp_o_f32_e64 s[10:11], v4, v20
+; GFX8-NEXT: v_cndmask_b32_e32 v3, v16, v3, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v4, v20
; GFX8-NEXT: v_max_f32_e32 v4, v4, v20
-; GFX8-NEXT: v_cmp_o_f32_e64 s[12:13], v5, v21
+; GFX8-NEXT: v_cndmask_b32_e32 v4, v16, v4, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v5, v21
; GFX8-NEXT: v_max_f32_e32 v5, v5, v21
-; GFX8-NEXT: v_cmp_o_f32_e64 s[14:15], v6, v22
+; GFX8-NEXT: v_cndmask_b32_e32 v5, v16, v5, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v6, v22
; GFX8-NEXT: v_max_f32_e32 v6, v6, v22
-; GFX8-NEXT: v_cmp_o_f32_e64 s[16:17], v7, v23
+; GFX8-NEXT: v_cndmask_b32_e32 v6, v16, v6, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v7, v23
; GFX8-NEXT: v_max_f32_e32 v7, v7, v23
-; GFX8-NEXT: v_cmp_o_f32_e64 s[18:19], v8, v24
+; GFX8-NEXT: v_cndmask_b32_e32 v7, v16, v7, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
; GFX8-NEXT: v_max_f32_e32 v8, v8, v24
-; GFX8-NEXT: v_cmp_o_f32_e64 s[20:21], v9, v25
+; GFX8-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
; GFX8-NEXT: v_max_f32_e32 v9, v9, v25
-; GFX8-NEXT: v_cmp_o_f32_e64 s[22:23], v10, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
; GFX8-NEXT: v_max_f32_e32 v10, v10, v26
-; GFX8-NEXT: v_cmp_o_f32_e64 s[24:25], v11, v27
+; GFX8-NEXT: v_cndmask_b32_e32 v10, v16, v10, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
; GFX8-NEXT: v_max_f32_e32 v11, v11, v27
-; GFX8-NEXT: v_cmp_o_f32_e64 s[26:27], v12, v28
+; GFX8-NEXT: v_cndmask_b32_e32 v11, v16, v11, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
; GFX8-NEXT: v_max_f32_e32 v12, v12, v28
-; GFX8-NEXT: v_max_f32_e32 v19, v14, v30
-; GFX8-NEXT: v_cmp_o_f32_e64 s[40:41], v14, v30
-; GFX8-NEXT: v_cndmask_b32_e32 v0, v17, v0, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v13, v17, v18, s[28:29]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v17, v1, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[16:17]
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[18:19]
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[20:21]
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[22:23]
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[24:25]
-; GFX8-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[26:27]
-; GFX8-NEXT: v_cndmask_b32_e64 v14, v17, v19, s[40:41]
+; GFX8-NEXT: v_cndmask_b32_e32 v12, v16, v12, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
+; GFX8-NEXT: v_max_f32_e32 v13, v13, v29
+; GFX8-NEXT: v_cndmask_b32_e32 v13, v16, v13, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX8-NEXT: v_max_f32_e32 v14, v14, v30
+; GFX8-NEXT: v_cndmask_b32_e32 v14, v16, v14, vcc
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_max_f32_e32 v18, v15, v16
-; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v15, v16
-; GFX8-NEXT: v_cndmask_b32_e32 v15, v17, v18, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX8-NEXT: v_max_f32_e32 v15, v15, v31
+; GFX8-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX900-LABEL: v_maximum_v16f32:
; GFX900: ; %bb.0:
; GFX900-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX900-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
; GFX900-NEXT: v_max_f32_e32 v0, v0, v16
-; GFX900-NEXT: buffer_load_dword v16, off, s[0:3], s32
-; GFX900-NEXT: v_cmp_o_f32_e64 s[4:5], v1, v17
+; GFX900-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX900-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
; GFX900-NEXT: v_max_f32_e32 v1, v1, v17
-; GFX900-NEXT: v_cmp_o_f32_e64 s[6:7], v2, v18
+; GFX900-NEXT: v_cndmask_b32_e32 v1, v16, v1, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v2, v18
; GFX900-NEXT: v_max_f32_e32 v2, v2, v18
-; GFX900-NEXT: v_mov_b32_e32 v17, 0x7fc00000
-; GFX900-NEXT: v_max_f32_e32 v18, v13, v29
-; GFX900-NEXT: v_cmp_o_f32_e64 s[28:29], v13, v29
-; GFX900-NEXT: v_cmp_o_f32_e64 s[8:9], v3, v19
+; GFX900-NEXT: v_cndmask_b32_e32 v2, v16, v2, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v3, v19
; GFX900-NEXT: v_max_f32_e32 v3, v3, v19
-; GFX900-NEXT: v_cmp_o_f32_e64 s[10:11], v4, v20
+; GFX900-NEXT: v_cndmask_b32_e32 v3, v16, v3, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v4, v20
; GFX900-NEXT: v_max_f32_e32 v4, v4, v20
-; GFX900-NEXT: v_cmp_o_f32_e64 s[12:13], v5, v21
+; GFX900-NEXT: v_cndmask_b32_e32 v4, v16, v4, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v5, v21
; GFX900-NEXT: v_max_f32_e32 v5, v5, v21
-; GFX900-NEXT: v_cmp_o_f32_e64 s[14:15], v6, v22
+; GFX900-NEXT: v_cndmask_b32_e32 v5, v16, v5, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v6, v22
; GFX900-NEXT: v_max_f32_e32 v6, v6, v22
-; GFX900-NEXT: v_cmp_o_f32_e64 s[16:17], v7, v23
+; GFX900-NEXT: v_cndmask_b32_e32 v6, v16, v6, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v7, v23
; GFX900-NEXT: v_max_f32_e32 v7, v7, v23
-; GFX900-NEXT: v_cmp_o_f32_e64 s[18:19], v8, v24
+; GFX900-NEXT: v_cndmask_b32_e32 v7, v16, v7, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
; GFX900-NEXT: v_max_f32_e32 v8, v8, v24
-; GFX900-NEXT: v_cmp_o_f32_e64 s[20:21], v9, v25
+; GFX900-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
; GFX900-NEXT: v_max_f32_e32 v9, v9, v25
-; GFX900-NEXT: v_cmp_o_f32_e64 s[22:23], v10, v26
+; GFX900-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
; GFX900-NEXT: v_max_f32_e32 v10, v10, v26
-; GFX900-NEXT: v_cmp_o_f32_e64 s[24:25], v11, v27
+; GFX900-NEXT: v_cndmask_b32_e32 v10, v16, v10, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
; GFX900-NEXT: v_max_f32_e32 v11, v11, v27
-; GFX900-NEXT: v_cmp_o_f32_e64 s[26:27], v12, v28
+; GFX900-NEXT: v_cndmask_b32_e32 v11, v16, v11, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
; GFX900-NEXT: v_max_f32_e32 v12, v12, v28
-; GFX900-NEXT: v_max_f32_e32 v19, v14, v30
-; GFX900-NEXT: v_cmp_o_f32_e64 s[40:41], v14, v30
-; GFX900-NEXT: v_cndmask_b32_e32 v0, v17, v0, vcc
-; GFX900-NEXT: v_cndmask_b32_e64 v13, v17, v18, s[28:29]
-; GFX900-NEXT: v_cndmask_b32_e64 v1, v17, v1, s[4:5]
-; GFX900-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[6:7]
-; GFX900-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[8:9]
-; GFX900-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[10:11]
-; GFX900-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[12:13]
-; GFX900-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[14:15]
-; GFX900-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[16:17]
-; GFX900-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[18:19]
-; GFX900-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[20:21]
-; GFX900-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[22:23]
-; GFX900-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[24:25]
-; GFX900-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[26:27]
-; GFX900-NEXT: v_cndmask_b32_e64 v14, v17, v19, s[40:41]
+; GFX900-NEXT: v_cndmask_b32_e32 v12, v16, v12, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
+; GFX900-NEXT: v_max_f32_e32 v13, v13, v29
+; GFX900-NEXT: v_cndmask_b32_e32 v13, v16, v13, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX900-NEXT: v_max_f32_e32 v14, v14, v30
+; GFX900-NEXT: v_cndmask_b32_e32 v14, v16, v14, vcc
; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_max_f32_e32 v18, v15, v16
-; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v15, v16
-; GFX900-NEXT: v_cndmask_b32_e32 v15, v17, v18, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX900-NEXT: v_max_f32_e32 v15, v15, v31
+; GFX900-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
; GFX900-NEXT: s_setpc_b64 s[30:31]
;
; GFX950-LABEL: v_maximum_v16f32:
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.maximum.f64.ll b/llvm/test/CodeGen/AMDGPU/llvm.maximum.f64.ll
index 3ed09b1589fb0..934fecf81be58 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.maximum.f64.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.maximum.f64.ll
@@ -1898,120 +1898,120 @@ define <8 x double> @v_maximum_v8f64(<8 x double> %src0, <8 x double> %src1) #0
; GFX7: ; %bb.0:
; GFX7-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX7-NEXT: v_max_f64 v[32:33], v[0:1], v[16:17]
; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[16:17]
-; GFX7-NEXT: v_max_f64 v[16:17], v[2:3], v[18:19]
+; GFX7-NEXT: v_max_f64 v[0:1], v[0:1], v[16:17]
; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[18:19]
-; GFX7-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX7-NEXT: v_max_f64 v[18:19], v[4:5], v[20:21]
+; GFX7-NEXT: v_max_f64 v[2:3], v[2:3], v[18:19]
; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
-; GFX7-NEXT: v_max_f64 v[20:21], v[6:7], v[22:23]
+; GFX7-NEXT: v_max_f64 v[4:5], v[4:5], v[20:21]
; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[6:7], v[22:23]
-; GFX7-NEXT: v_max_f64 v[22:23], v[8:9], v[24:25]
+; GFX7-NEXT: v_max_f64 v[6:7], v[6:7], v[22:23]
; GFX7-NEXT: v_cmp_u_f64_e64 s[10:11], v[8:9], v[24:25]
-; GFX7-NEXT: v_max_f64 v[24:25], v[10:11], v[26:27]
+; GFX7-NEXT: v_max_f64 v[8:9], v[8:9], v[24:25]
; GFX7-NEXT: v_cmp_u_f64_e64 s[12:13], v[10:11], v[26:27]
-; GFX7-NEXT: v_max_f64 v[26:27], v[12:13], v[28:29]
+; GFX7-NEXT: v_max_f64 v[10:11], v[10:11], v[26:27]
; GFX7-NEXT: v_cmp_u_f64_e64 s[14:15], v[12:13], v[28:29]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v32, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e32 v1, v33, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v16, 0, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v3, v17, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v18, 0, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v19, v34, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v20, 0, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v21, v34, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v8, v22, 0, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v9, v23, v34, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v10, v24, 0, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v11, v25, v34, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v12, v26, 0, s[14:15]
-; GFX7-NEXT: v_cndmask_b32_e64 v13, v27, v34, s[14:15]
+; GFX7-NEXT: v_max_f64 v[12:13], v[12:13], v[28:29]
+; GFX7-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v9, v9, v16, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[12:13]
+; GFX7-NEXT: v_cndmask_b32_e64 v11, v11, v16, s[12:13]
+; GFX7-NEXT: v_cndmask_b32_e64 v12, v12, 0, s[14:15]
+; GFX7-NEXT: v_cndmask_b32_e64 v13, v13, v16, s[14:15]
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: v_max_f64 v[16:17], v[14:15], v[30:31]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[14:15], v[30:31]
-; GFX7-NEXT: v_cndmask_b32_e64 v14, v16, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e32 v15, v17, v34, vcc
+; GFX7-NEXT: v_cmp_u_f64_e64 s[16:17], v[14:15], v[30:31]
+; GFX7-NEXT: v_max_f64 v[14:15], v[14:15], v[30:31]
+; GFX7-NEXT: v_cndmask_b32_e64 v15, v15, v16, s[16:17]
+; GFX7-NEXT: v_cndmask_b32_e64 v14, v14, 0, s[16:17]
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: v_maximum_v8f64:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX8-NEXT: v_max_f64 v[32:33], v[0:1], v[16:17]
; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[16:17]
-; GFX8-NEXT: v_max_f64 v[16:17], v[2:3], v[18:19]
+; GFX8-NEXT: v_max_f64 v[0:1], v[0:1], v[16:17]
; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[18:19]
-; GFX8-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX8-NEXT: v_max_f64 v[18:19], v[4:5], v[20:21]
+; GFX8-NEXT: v_max_f64 v[2:3], v[2:3], v[18:19]
; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
-; GFX8-NEXT: v_max_f64 v[20:21], v[6:7], v[22:23]
+; GFX8-NEXT: v_max_f64 v[4:5], v[4:5], v[20:21]
; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[6:7], v[22:23]
-; GFX8-NEXT: v_max_f64 v[22:23], v[8:9], v[24:25]
+; GFX8-NEXT: v_max_f64 v[6:7], v[6:7], v[22:23]
; GFX8-NEXT: v_cmp_u_f64_e64 s[10:11], v[8:9], v[24:25]
-; GFX8-NEXT: v_max_f64 v[24:25], v[10:11], v[26:27]
+; GFX8-NEXT: v_max_f64 v[8:9], v[8:9], v[24:25]
; GFX8-NEXT: v_cmp_u_f64_e64 s[12:13], v[10:11], v[26:27]
-; GFX8-NEXT: v_max_f64 v[26:27], v[12:13], v[28:29]
+; GFX8-NEXT: v_max_f64 v[10:11], v[10:11], v[26:27]
; GFX8-NEXT: v_cmp_u_f64_e64 s[14:15], v[12:13], v[28:29]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v32, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v33, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v16, 0, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v17, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v18, 0, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v19, v34, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v20, 0, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v21, v34, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v22, 0, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v23, v34, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v24, 0, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v25, v34, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v12, v26, 0, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e64 v13, v27, v34, s[14:15]
+; GFX8-NEXT: v_max_f64 v[12:13], v[12:13], v[28:29]
+; GFX8-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v9, v9, v16, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[12:13]
+; GFX8-NEXT: v_cndmask_b32_e64 v11, v11, v16, s[12:13]
+; GFX8-NEXT: v_cndmask_b32_e64 v12, v12, 0, s[14:15]
+; GFX8-NEXT: v_cndmask_b32_e64 v13, v13, v16, s[14:15]
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_max_f64 v[16:17], v[14:15], v[30:31]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[14:15], v[30:31]
-; GFX8-NEXT: v_cndmask_b32_e64 v14, v16, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v15, v17, v34, vcc
+; GFX8-NEXT: v_cmp_u_f64_e64 s[16:17], v[14:15], v[30:31]
+; GFX8-NEXT: v_max_f64 v[14:15], v[14:15], v[30:31]
+; GFX8-NEXT: v_cndmask_b32_e64 v15, v15, v16, s[16:17]
+; GFX8-NEXT: v_cndmask_b32_e64 v14, v14, 0, s[16:17]
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX900-LABEL: v_maximum_v8f64:
; GFX900: ; %bb.0:
; GFX900-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX900-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX900-NEXT: v_max_f64 v[32:33], v[0:1], v[16:17]
; GFX900-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[16:17]
-; GFX900-NEXT: v_max_f64 v[16:17], v[2:3], v[18:19]
+; GFX900-NEXT: v_max_f64 v[0:1], v[0:1], v[16:17]
; GFX900-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[18:19]
-; GFX900-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX900-NEXT: v_max_f64 v[18:19], v[4:5], v[20:21]
+; GFX900-NEXT: v_max_f64 v[2:3], v[2:3], v[18:19]
; GFX900-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
-; GFX900-NEXT: v_max_f64 v[20:21], v[6:7], v[22:23]
+; GFX900-NEXT: v_max_f64 v[4:5], v[4:5], v[20:21]
; GFX900-NEXT: v_cmp_u_f64_e64 s[8:9], v[6:7], v[22:23]
-; GFX900-NEXT: v_max_f64 v[22:23], v[8:9], v[24:25]
+; GFX900-NEXT: v_max_f64 v[6:7], v[6:7], v[22:23]
; GFX900-NEXT: v_cmp_u_f64_e64 s[10:11], v[8:9], v[24:25]
-; GFX900-NEXT: v_max_f64 v[24:25], v[10:11], v[26:27]
+; GFX900-NEXT: v_max_f64 v[8:9], v[8:9], v[24:25]
; GFX900-NEXT: v_cmp_u_f64_e64 s[12:13], v[10:11], v[26:27]
-; GFX900-NEXT: v_max_f64 v[26:27], v[12:13], v[28:29]
+; GFX900-NEXT: v_max_f64 v[10:11], v[10:11], v[26:27]
; GFX900-NEXT: v_cmp_u_f64_e64 s[14:15], v[12:13], v[28:29]
-; GFX900-NEXT: v_cndmask_b32_e64 v0, v32, 0, vcc
-; GFX900-NEXT: v_cndmask_b32_e32 v1, v33, v34, vcc
-; GFX900-NEXT: v_cndmask_b32_e64 v2, v16, 0, s[4:5]
-; GFX900-NEXT: v_cndmask_b32_e64 v3, v17, v34, s[4:5]
-; GFX900-NEXT: v_cndmask_b32_e64 v4, v18, 0, s[6:7]
-; GFX900-NEXT: v_cndmask_b32_e64 v5, v19, v34, s[6:7]
-; GFX900-NEXT: v_cndmask_b32_e64 v6, v20, 0, s[8:9]
-; GFX900-NEXT: v_cndmask_b32_e64 v7, v21, v34, s[8:9]
-; GFX900-NEXT: v_cndmask_b32_e64 v8, v22, 0, s[10:11]
-; GFX900-NEXT: v_cndmask_b32_e64 v9, v23, v34, s[10:11]
-; GFX900-NEXT: v_cndmask_b32_e64 v10, v24, 0, s[12:13]
-; GFX900-NEXT: v_cndmask_b32_e64 v11, v25, v34, s[12:13]
-; GFX900-NEXT: v_cndmask_b32_e64 v12, v26, 0, s[14:15]
-; GFX900-NEXT: v_cndmask_b32_e64 v13, v27, v34, s[14:15]
+; GFX900-NEXT: v_max_f64 v[12:13], v[12:13], v[28:29]
+; GFX900-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX900-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX900-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX900-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
+; GFX900-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX900-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX900-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX900-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[8:9]
+; GFX900-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[8:9]
+; GFX900-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[10:11]
+; GFX900-NEXT: v_cndmask_b32_e64 v9, v9, v16, s[10:11]
+; GFX900-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[12:13]
+; GFX900-NEXT: v_cndmask_b32_e64 v11, v11, v16, s[12:13]
+; GFX900-NEXT: v_cndmask_b32_e64 v12, v12, 0, s[14:15]
+; GFX900-NEXT: v_cndmask_b32_e64 v13, v13, v16, s[14:15]
; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_max_f64 v[16:17], v[14:15], v[30:31]
-; GFX900-NEXT: v_cmp_u_f64_e32 vcc, v[14:15], v[30:31]
-; GFX900-NEXT: v_cndmask_b32_e64 v14, v16, 0, vcc
-; GFX900-NEXT: v_cndmask_b32_e32 v15, v17, v34, vcc
+; GFX900-NEXT: v_cmp_u_f64_e64 s[16:17], v[14:15], v[30:31]
+; GFX900-NEXT: v_max_f64 v[14:15], v[14:15], v[30:31]
+; GFX900-NEXT: v_cndmask_b32_e64 v15, v15, v16, s[16:17]
+; GFX900-NEXT: v_cndmask_b32_e64 v14, v14, 0, s[16:17]
; GFX900-NEXT: s_setpc_b64 s[30:31]
;
; GFX950-LABEL: v_maximum_v8f64:
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.minimum.f16.ll b/llvm/test/CodeGen/AMDGPU/llvm.minimum.f16.ll
index 48a36737ce45a..481f96cf258f7 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.minimum.f16.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.minimum.f16.ll
@@ -1978,87 +1978,87 @@ define <16 x half> @v_minimum_v16f16(<16 x half> %src0, <16 x half> %src1) {
; GFX8-LABEL: v_minimum_v16f16:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v14
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v6
-; GFX8-NEXT: v_min_f16_e32 v16, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v18, v17
+; GFX8-NEXT: v_lshrrev_b32_e32 v16, 16, v14
+; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v6
+; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v17, v16
+; GFX8-NEXT: v_min_f16_e32 v16, v17, v16
; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v13
; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v5
-; GFX8-NEXT: v_min_f16_e32 v20, v18, v17
; GFX8-NEXT: v_cmp_o_f16_e64 s[4:5], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v12
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v4
-; GFX8-NEXT: v_min_f16_e32 v21, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[6:7], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v11
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v3
-; GFX8-NEXT: v_min_f16_e32 v22, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[8:9], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v10
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v2
-; GFX8-NEXT: v_min_f16_e32 v23, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[10:11], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v9
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v1
-; GFX8-NEXT: v_mov_b32_e32 v19, 0x7e00
-; GFX8-NEXT: v_min_f16_e32 v24, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[12:13], v18, v17
-; GFX8-NEXT: v_lshrrev_b32_e32 v17, 16, v8
-; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v0
-; GFX8-NEXT: v_min_f16_e32 v25, v18, v17
-; GFX8-NEXT: v_cmp_o_f16_e64 s[14:15], v18, v17
-; GFX8-NEXT: v_min_f16_e32 v17, v6, v14
+; GFX8-NEXT: v_min_f16_e32 v17, v18, v17
+; GFX8-NEXT: v_lshrrev_b32_e32 v18, 16, v12
+; GFX8-NEXT: v_lshrrev_b32_e32 v19, 16, v4
+; GFX8-NEXT: v_cmp_o_f16_e64 s[6:7], v19, v18
+; GFX8-NEXT: v_min_f16_e32 v18, v19, v18
+; GFX8-NEXT: v_lshrrev_b32_e32 v19, 16, v11
+; GFX8-NEXT: v_lshrrev_b32_e32 v20, 16, v3
+; GFX8-NEXT: v_cmp_o_f16_e64 s[8:9], v20, v19
+; GFX8-NEXT: v_min_f16_e32 v19, v20, v19
+; GFX8-NEXT: v_lshrrev_b32_e32 v20, 16, v10
+; GFX8-NEXT: v_lshrrev_b32_e32 v21, 16, v2
+; GFX8-NEXT: v_cmp_o_f16_e64 s[10:11], v21, v20
+; GFX8-NEXT: v_min_f16_e32 v20, v21, v20
+; GFX8-NEXT: v_lshrrev_b32_e32 v21, 16, v9
+; GFX8-NEXT: v_lshrrev_b32_e32 v22, 16, v1
+; GFX8-NEXT: v_cmp_o_f16_e64 s[12:13], v22, v21
+; GFX8-NEXT: v_min_f16_e32 v21, v22, v21
+; GFX8-NEXT: v_lshrrev_b32_e32 v22, 16, v8
+; GFX8-NEXT: v_lshrrev_b32_e32 v23, 16, v0
+; GFX8-NEXT: v_cmp_o_f16_e64 s[14:15], v23, v22
+; GFX8-NEXT: v_min_f16_e32 v22, v23, v22
+; GFX8-NEXT: v_cmp_o_f16_e64 s[22:23], v3, v11
+; GFX8-NEXT: v_min_f16_e32 v3, v3, v11
+; GFX8-NEXT: v_cmp_o_f16_e64 s[24:25], v2, v10
+; GFX8-NEXT: v_min_f16_e32 v2, v2, v10
+; GFX8-NEXT: v_cmp_o_f16_e64 s[26:27], v7, v15
+; GFX8-NEXT: v_min_f16_e32 v10, v7, v15
+; GFX8-NEXT: v_lshrrev_b32_e32 v11, 16, v15
+; GFX8-NEXT: v_lshrrev_b32_e32 v7, 16, v7
+; GFX8-NEXT: v_cmp_o_f16_e64 s[42:43], v0, v8
+; GFX8-NEXT: v_min_f16_e32 v0, v0, v8
+; GFX8-NEXT: v_mov_b32_e32 v8, 0x7e00
; GFX8-NEXT: v_cmp_o_f16_e64 s[16:17], v6, v14
-; GFX8-NEXT: v_min_f16_e32 v6, v5, v13
+; GFX8-NEXT: v_min_f16_e32 v6, v6, v14
; GFX8-NEXT: v_cmp_o_f16_e64 s[18:19], v5, v13
-; GFX8-NEXT: v_min_f16_e32 v5, v4, v12
+; GFX8-NEXT: v_min_f16_e32 v5, v5, v13
; GFX8-NEXT: v_cmp_o_f16_e64 s[20:21], v4, v12
-; GFX8-NEXT: v_min_f16_e32 v4, v3, v11
-; GFX8-NEXT: v_cmp_o_f16_e64 s[22:23], v3, v11
-; GFX8-NEXT: v_min_f16_e32 v3, v2, v10
-; GFX8-NEXT: v_min_f16_e32 v11, v7, v15
-; GFX8-NEXT: v_cmp_o_f16_e64 s[24:25], v7, v15
-; GFX8-NEXT: v_lshrrev_b32_e32 v12, 16, v15
-; GFX8-NEXT: v_lshrrev_b32_e32 v7, 16, v7
-; GFX8-NEXT: v_cndmask_b32_e32 v14, v19, v16, vcc
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v2, v10
-; GFX8-NEXT: v_min_f16_e32 v13, v7, v12
-; GFX8-NEXT: v_cmp_o_f16_e64 s[26:27], v7, v12
-; GFX8-NEXT: v_min_f16_e32 v7, v1, v9
-; GFX8-NEXT: v_cndmask_b32_e32 v2, v19, v3, vcc
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v1, v9
-; GFX8-NEXT: v_min_f16_e32 v12, v0, v8
-; GFX8-NEXT: v_cndmask_b32_e64 v18, v19, v22, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v22, v19, v25, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v19, v7, vcc
-; GFX8-NEXT: v_cmp_o_f16_e32 vcc, v0, v8
-; GFX8-NEXT: v_cndmask_b32_e64 v16, v19, v21, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v21, v19, v24, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e32 v0, v19, v12, vcc
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v22
-; GFX8-NEXT: v_cndmask_b32_e64 v15, v19, v20, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v20, v19, v23, s[10:11]
-; GFX8-NEXT: v_or_b32_sdwa v0, v0, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v21
-; GFX8-NEXT: v_or_b32_sdwa v1, v1, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v20
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v19, v4, s[22:23]
-; GFX8-NEXT: v_or_b32_sdwa v2, v2, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v3, 16, v18
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v19, v5, s[20:21]
-; GFX8-NEXT: v_or_b32_sdwa v3, v4, v3 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v4, 16, v16
-; GFX8-NEXT: v_cndmask_b32_e64 v13, v19, v13, s[26:27]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v19, v6, s[18:19]
-; GFX8-NEXT: v_or_b32_sdwa v4, v5, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v5, 16, v15
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v19, v11, s[24:25]
-; GFX8-NEXT: v_cndmask_b32_e64 v17, v19, v17, s[16:17]
-; GFX8-NEXT: v_or_b32_sdwa v5, v6, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_lshlrev_b32_e32 v6, 16, v14
-; GFX8-NEXT: v_lshlrev_b32_e32 v7, 16, v13
-; GFX8-NEXT: v_or_b32_sdwa v6, v17, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
-; GFX8-NEXT: v_or_b32_sdwa v7, v11, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_min_f16_e32 v4, v4, v12
+; GFX8-NEXT: v_cmp_o_f16_e64 s[28:29], v7, v11
+; GFX8-NEXT: v_min_f16_e32 v7, v7, v11
+; GFX8-NEXT: v_cmp_o_f16_e64 s[40:41], v1, v9
+; GFX8-NEXT: v_min_f16_e32 v1, v1, v9
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v8, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v16, v8, v22, s[14:15]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v8, v7, s[28:29]
+; GFX8-NEXT: v_cndmask_b32_e64 v11, v8, v17, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v12, v8, v18, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v13, v8, v19, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v14, v8, v20, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v15, v8, v21, s[12:13]
+; GFX8-NEXT: v_cndmask_b32_e64 v10, v8, v10, s[26:27]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v8, v6, s[16:17]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v8, v5, s[18:19]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v8, v4, s[20:21]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v8, v3, s[22:23]
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v8, v2, s[24:25]
+; GFX8-NEXT: v_cndmask_b32_e64 v1, v8, v1, s[40:41]
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v8, v0, s[42:43]
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v16
+; GFX8-NEXT: v_or_b32_sdwa v0, v0, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v15
+; GFX8-NEXT: v_or_b32_sdwa v1, v1, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v14
+; GFX8-NEXT: v_or_b32_sdwa v2, v2, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v13
+; GFX8-NEXT: v_or_b32_sdwa v3, v3, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v12
+; GFX8-NEXT: v_or_b32_sdwa v4, v4, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v11
+; GFX8-NEXT: v_or_b32_sdwa v5, v5, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_lshlrev_b32_e32 v8, 16, v9
+; GFX8-NEXT: v_lshlrev_b32_e32 v7, 16, v7
+; GFX8-NEXT: v_or_b32_sdwa v6, v6, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
+; GFX8-NEXT: v_or_b32_sdwa v7, v10, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX900-LABEL: v_minimum_v16f16:
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.minimum.f32.ll b/llvm/test/CodeGen/AMDGPU/llvm.minimum.f32.ll
index a579a4be07a76..25733cbe8d4eb 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.minimum.f32.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.minimum.f32.ll
@@ -1836,169 +1836,169 @@ define <16 x float> @v_minimum_v16f32(<16 x float> %src0, <16 x float> %src1) {
; GFX7-LABEL: v_minimum_v16f32:
; GFX7: ; %bb.0:
; GFX7-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
; GFX7-NEXT: v_min_f32_e32 v0, v0, v16
-; GFX7-NEXT: buffer_load_dword v16, off, s[0:3], s32
-; GFX7-NEXT: v_cmp_o_f32_e64 s[4:5], v1, v17
+; GFX7-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX7-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
; GFX7-NEXT: v_min_f32_e32 v1, v1, v17
-; GFX7-NEXT: v_cmp_o_f32_e64 s[6:7], v2, v18
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v16, v1, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v2, v18
; GFX7-NEXT: v_min_f32_e32 v2, v2, v18
-; GFX7-NEXT: v_mov_b32_e32 v17, 0x7fc00000
-; GFX7-NEXT: v_min_f32_e32 v18, v13, v29
-; GFX7-NEXT: v_cmp_o_f32_e64 s[28:29], v13, v29
-; GFX7-NEXT: v_cmp_o_f32_e64 s[8:9], v3, v19
+; GFX7-NEXT: v_cndmask_b32_e32 v2, v16, v2, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v3, v19
; GFX7-NEXT: v_min_f32_e32 v3, v3, v19
-; GFX7-NEXT: v_cmp_o_f32_e64 s[10:11], v4, v20
+; GFX7-NEXT: v_cndmask_b32_e32 v3, v16, v3, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v4, v20
; GFX7-NEXT: v_min_f32_e32 v4, v4, v20
-; GFX7-NEXT: v_cmp_o_f32_e64 s[12:13], v5, v21
+; GFX7-NEXT: v_cndmask_b32_e32 v4, v16, v4, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v5, v21
; GFX7-NEXT: v_min_f32_e32 v5, v5, v21
-; GFX7-NEXT: v_cmp_o_f32_e64 s[14:15], v6, v22
+; GFX7-NEXT: v_cndmask_b32_e32 v5, v16, v5, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v6, v22
; GFX7-NEXT: v_min_f32_e32 v6, v6, v22
-; GFX7-NEXT: v_cmp_o_f32_e64 s[16:17], v7, v23
+; GFX7-NEXT: v_cndmask_b32_e32 v6, v16, v6, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v7, v23
; GFX7-NEXT: v_min_f32_e32 v7, v7, v23
-; GFX7-NEXT: v_cmp_o_f32_e64 s[18:19], v8, v24
+; GFX7-NEXT: v_cndmask_b32_e32 v7, v16, v7, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
; GFX7-NEXT: v_min_f32_e32 v8, v8, v24
-; GFX7-NEXT: v_cmp_o_f32_e64 s[20:21], v9, v25
+; GFX7-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
; GFX7-NEXT: v_min_f32_e32 v9, v9, v25
-; GFX7-NEXT: v_cmp_o_f32_e64 s[22:23], v10, v26
+; GFX7-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
; GFX7-NEXT: v_min_f32_e32 v10, v10, v26
-; GFX7-NEXT: v_cmp_o_f32_e64 s[24:25], v11, v27
+; GFX7-NEXT: v_cndmask_b32_e32 v10, v16, v10, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
; GFX7-NEXT: v_min_f32_e32 v11, v11, v27
-; GFX7-NEXT: v_cmp_o_f32_e64 s[26:27], v12, v28
+; GFX7-NEXT: v_cndmask_b32_e32 v11, v16, v11, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
; GFX7-NEXT: v_min_f32_e32 v12, v12, v28
-; GFX7-NEXT: v_min_f32_e32 v19, v14, v30
-; GFX7-NEXT: v_cmp_o_f32_e64 s[40:41], v14, v30
-; GFX7-NEXT: v_cndmask_b32_e32 v0, v17, v0, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v13, v17, v18, s[28:29]
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v17, v1, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[14:15]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[16:17]
-; GFX7-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[18:19]
-; GFX7-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[20:21]
-; GFX7-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[22:23]
-; GFX7-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[24:25]
-; GFX7-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[26:27]
-; GFX7-NEXT: v_cndmask_b32_e64 v14, v17, v19, s[40:41]
+; GFX7-NEXT: v_cndmask_b32_e32 v12, v16, v12, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
+; GFX7-NEXT: v_min_f32_e32 v13, v13, v29
+; GFX7-NEXT: v_cndmask_b32_e32 v13, v16, v13, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX7-NEXT: v_min_f32_e32 v14, v14, v30
+; GFX7-NEXT: v_cndmask_b32_e32 v14, v16, v14, vcc
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: v_min_f32_e32 v18, v15, v16
-; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v15, v16
-; GFX7-NEXT: v_cndmask_b32_e32 v15, v17, v18, vcc
+; GFX7-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX7-NEXT: v_min_f32_e32 v15, v15, v31
+; GFX7-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: v_minimum_v16f32:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX8-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
; GFX8-NEXT: v_min_f32_e32 v0, v0, v16
-; GFX8-NEXT: buffer_load_dword v16, off, s[0:3], s32
-; GFX8-NEXT: v_cmp_o_f32_e64 s[4:5], v1, v17
+; GFX8-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX8-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
; GFX8-NEXT: v_min_f32_e32 v1, v1, v17
-; GFX8-NEXT: v_cmp_o_f32_e64 s[6:7], v2, v18
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v16, v1, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v2, v18
; GFX8-NEXT: v_min_f32_e32 v2, v2, v18
-; GFX8-NEXT: v_mov_b32_e32 v17, 0x7fc00000
-; GFX8-NEXT: v_min_f32_e32 v18, v13, v29
-; GFX8-NEXT: v_cmp_o_f32_e64 s[28:29], v13, v29
-; GFX8-NEXT: v_cmp_o_f32_e64 s[8:9], v3, v19
+; GFX8-NEXT: v_cndmask_b32_e32 v2, v16, v2, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v3, v19
; GFX8-NEXT: v_min_f32_e32 v3, v3, v19
-; GFX8-NEXT: v_cmp_o_f32_e64 s[10:11], v4, v20
+; GFX8-NEXT: v_cndmask_b32_e32 v3, v16, v3, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v4, v20
; GFX8-NEXT: v_min_f32_e32 v4, v4, v20
-; GFX8-NEXT: v_cmp_o_f32_e64 s[12:13], v5, v21
+; GFX8-NEXT: v_cndmask_b32_e32 v4, v16, v4, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v5, v21
; GFX8-NEXT: v_min_f32_e32 v5, v5, v21
-; GFX8-NEXT: v_cmp_o_f32_e64 s[14:15], v6, v22
+; GFX8-NEXT: v_cndmask_b32_e32 v5, v16, v5, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v6, v22
; GFX8-NEXT: v_min_f32_e32 v6, v6, v22
-; GFX8-NEXT: v_cmp_o_f32_e64 s[16:17], v7, v23
+; GFX8-NEXT: v_cndmask_b32_e32 v6, v16, v6, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v7, v23
; GFX8-NEXT: v_min_f32_e32 v7, v7, v23
-; GFX8-NEXT: v_cmp_o_f32_e64 s[18:19], v8, v24
+; GFX8-NEXT: v_cndmask_b32_e32 v7, v16, v7, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
; GFX8-NEXT: v_min_f32_e32 v8, v8, v24
-; GFX8-NEXT: v_cmp_o_f32_e64 s[20:21], v9, v25
+; GFX8-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
; GFX8-NEXT: v_min_f32_e32 v9, v9, v25
-; GFX8-NEXT: v_cmp_o_f32_e64 s[22:23], v10, v26
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
; GFX8-NEXT: v_min_f32_e32 v10, v10, v26
-; GFX8-NEXT: v_cmp_o_f32_e64 s[24:25], v11, v27
+; GFX8-NEXT: v_cndmask_b32_e32 v10, v16, v10, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
; GFX8-NEXT: v_min_f32_e32 v11, v11, v27
-; GFX8-NEXT: v_cmp_o_f32_e64 s[26:27], v12, v28
+; GFX8-NEXT: v_cndmask_b32_e32 v11, v16, v11, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
; GFX8-NEXT: v_min_f32_e32 v12, v12, v28
-; GFX8-NEXT: v_min_f32_e32 v19, v14, v30
-; GFX8-NEXT: v_cmp_o_f32_e64 s[40:41], v14, v30
-; GFX8-NEXT: v_cndmask_b32_e32 v0, v17, v0, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v13, v17, v18, s[28:29]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v17, v1, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[16:17]
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[18:19]
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[20:21]
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[22:23]
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[24:25]
-; GFX8-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[26:27]
-; GFX8-NEXT: v_cndmask_b32_e64 v14, v17, v19, s[40:41]
+; GFX8-NEXT: v_cndmask_b32_e32 v12, v16, v12, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
+; GFX8-NEXT: v_min_f32_e32 v13, v13, v29
+; GFX8-NEXT: v_cndmask_b32_e32 v13, v16, v13, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX8-NEXT: v_min_f32_e32 v14, v14, v30
+; GFX8-NEXT: v_cndmask_b32_e32 v14, v16, v14, vcc
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_min_f32_e32 v18, v15, v16
-; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v15, v16
-; GFX8-NEXT: v_cndmask_b32_e32 v15, v17, v18, vcc
+; GFX8-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX8-NEXT: v_min_f32_e32 v15, v15, v31
+; GFX8-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX900-LABEL: v_minimum_v16f32:
; GFX900: ; %bb.0:
; GFX900-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX900-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v0, v16
; GFX900-NEXT: v_min_f32_e32 v0, v0, v16
-; GFX900-NEXT: buffer_load_dword v16, off, s[0:3], s32
-; GFX900-NEXT: v_cmp_o_f32_e64 s[4:5], v1, v17
+; GFX900-NEXT: v_mov_b32_e32 v16, 0x7fc00000
+; GFX900-NEXT: v_cndmask_b32_e32 v0, v16, v0, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v1, v17
; GFX900-NEXT: v_min_f32_e32 v1, v1, v17
-; GFX900-NEXT: v_cmp_o_f32_e64 s[6:7], v2, v18
+; GFX900-NEXT: v_cndmask_b32_e32 v1, v16, v1, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v2, v18
; GFX900-NEXT: v_min_f32_e32 v2, v2, v18
-; GFX900-NEXT: v_mov_b32_e32 v17, 0x7fc00000
-; GFX900-NEXT: v_min_f32_e32 v18, v13, v29
-; GFX900-NEXT: v_cmp_o_f32_e64 s[28:29], v13, v29
-; GFX900-NEXT: v_cmp_o_f32_e64 s[8:9], v3, v19
+; GFX900-NEXT: v_cndmask_b32_e32 v2, v16, v2, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v3, v19
; GFX900-NEXT: v_min_f32_e32 v3, v3, v19
-; GFX900-NEXT: v_cmp_o_f32_e64 s[10:11], v4, v20
+; GFX900-NEXT: v_cndmask_b32_e32 v3, v16, v3, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v4, v20
; GFX900-NEXT: v_min_f32_e32 v4, v4, v20
-; GFX900-NEXT: v_cmp_o_f32_e64 s[12:13], v5, v21
+; GFX900-NEXT: v_cndmask_b32_e32 v4, v16, v4, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v5, v21
; GFX900-NEXT: v_min_f32_e32 v5, v5, v21
-; GFX900-NEXT: v_cmp_o_f32_e64 s[14:15], v6, v22
+; GFX900-NEXT: v_cndmask_b32_e32 v5, v16, v5, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v6, v22
; GFX900-NEXT: v_min_f32_e32 v6, v6, v22
-; GFX900-NEXT: v_cmp_o_f32_e64 s[16:17], v7, v23
+; GFX900-NEXT: v_cndmask_b32_e32 v6, v16, v6, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v7, v23
; GFX900-NEXT: v_min_f32_e32 v7, v7, v23
-; GFX900-NEXT: v_cmp_o_f32_e64 s[18:19], v8, v24
+; GFX900-NEXT: v_cndmask_b32_e32 v7, v16, v7, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v8, v24
; GFX900-NEXT: v_min_f32_e32 v8, v8, v24
-; GFX900-NEXT: v_cmp_o_f32_e64 s[20:21], v9, v25
+; GFX900-NEXT: v_cndmask_b32_e32 v8, v16, v8, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v9, v25
; GFX900-NEXT: v_min_f32_e32 v9, v9, v25
-; GFX900-NEXT: v_cmp_o_f32_e64 s[22:23], v10, v26
+; GFX900-NEXT: v_cndmask_b32_e32 v9, v16, v9, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v10, v26
; GFX900-NEXT: v_min_f32_e32 v10, v10, v26
-; GFX900-NEXT: v_cmp_o_f32_e64 s[24:25], v11, v27
+; GFX900-NEXT: v_cndmask_b32_e32 v10, v16, v10, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v11, v27
; GFX900-NEXT: v_min_f32_e32 v11, v11, v27
-; GFX900-NEXT: v_cmp_o_f32_e64 s[26:27], v12, v28
+; GFX900-NEXT: v_cndmask_b32_e32 v11, v16, v11, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v12, v28
; GFX900-NEXT: v_min_f32_e32 v12, v12, v28
-; GFX900-NEXT: v_min_f32_e32 v19, v14, v30
-; GFX900-NEXT: v_cmp_o_f32_e64 s[40:41], v14, v30
-; GFX900-NEXT: v_cndmask_b32_e32 v0, v17, v0, vcc
-; GFX900-NEXT: v_cndmask_b32_e64 v13, v17, v18, s[28:29]
-; GFX900-NEXT: v_cndmask_b32_e64 v1, v17, v1, s[4:5]
-; GFX900-NEXT: v_cndmask_b32_e64 v2, v17, v2, s[6:7]
-; GFX900-NEXT: v_cndmask_b32_e64 v3, v17, v3, s[8:9]
-; GFX900-NEXT: v_cndmask_b32_e64 v4, v17, v4, s[10:11]
-; GFX900-NEXT: v_cndmask_b32_e64 v5, v17, v5, s[12:13]
-; GFX900-NEXT: v_cndmask_b32_e64 v6, v17, v6, s[14:15]
-; GFX900-NEXT: v_cndmask_b32_e64 v7, v17, v7, s[16:17]
-; GFX900-NEXT: v_cndmask_b32_e64 v8, v17, v8, s[18:19]
-; GFX900-NEXT: v_cndmask_b32_e64 v9, v17, v9, s[20:21]
-; GFX900-NEXT: v_cndmask_b32_e64 v10, v17, v10, s[22:23]
-; GFX900-NEXT: v_cndmask_b32_e64 v11, v17, v11, s[24:25]
-; GFX900-NEXT: v_cndmask_b32_e64 v12, v17, v12, s[26:27]
-; GFX900-NEXT: v_cndmask_b32_e64 v14, v17, v19, s[40:41]
+; GFX900-NEXT: v_cndmask_b32_e32 v12, v16, v12, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v13, v29
+; GFX900-NEXT: v_min_f32_e32 v13, v13, v29
+; GFX900-NEXT: v_cndmask_b32_e32 v13, v16, v13, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v14, v30
+; GFX900-NEXT: v_min_f32_e32 v14, v14, v30
+; GFX900-NEXT: v_cndmask_b32_e32 v14, v16, v14, vcc
; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_min_f32_e32 v18, v15, v16
-; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v15, v16
-; GFX900-NEXT: v_cndmask_b32_e32 v15, v17, v18, vcc
+; GFX900-NEXT: v_cmp_o_f32_e32 vcc, v15, v31
+; GFX900-NEXT: v_min_f32_e32 v15, v15, v31
+; GFX900-NEXT: v_cndmask_b32_e32 v15, v16, v15, vcc
; GFX900-NEXT: s_setpc_b64 s[30:31]
;
; GFX950-LABEL: v_minimum_v16f32:
diff --git a/llvm/test/CodeGen/AMDGPU/llvm.minimum.f64.ll b/llvm/test/CodeGen/AMDGPU/llvm.minimum.f64.ll
index 2f18ad05ad3e2..792fbae6a0bc3 100644
--- a/llvm/test/CodeGen/AMDGPU/llvm.minimum.f64.ll
+++ b/llvm/test/CodeGen/AMDGPU/llvm.minimum.f64.ll
@@ -1898,120 +1898,120 @@ define <8 x double> @v_minimum_v8f64(<8 x double> %src0, <8 x double> %src1) #0
; GFX7: ; %bb.0:
; GFX7-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX7-NEXT: v_min_f64 v[32:33], v[0:1], v[16:17]
; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[16:17]
-; GFX7-NEXT: v_min_f64 v[16:17], v[2:3], v[18:19]
+; GFX7-NEXT: v_min_f64 v[0:1], v[0:1], v[16:17]
; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[18:19]
-; GFX7-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX7-NEXT: v_min_f64 v[18:19], v[4:5], v[20:21]
+; GFX7-NEXT: v_min_f64 v[2:3], v[2:3], v[18:19]
; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
-; GFX7-NEXT: v_min_f64 v[20:21], v[6:7], v[22:23]
+; GFX7-NEXT: v_min_f64 v[4:5], v[4:5], v[20:21]
; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[6:7], v[22:23]
-; GFX7-NEXT: v_min_f64 v[22:23], v[8:9], v[24:25]
+; GFX7-NEXT: v_min_f64 v[6:7], v[6:7], v[22:23]
; GFX7-NEXT: v_cmp_u_f64_e64 s[10:11], v[8:9], v[24:25]
-; GFX7-NEXT: v_min_f64 v[24:25], v[10:11], v[26:27]
+; GFX7-NEXT: v_min_f64 v[8:9], v[8:9], v[24:25]
; GFX7-NEXT: v_cmp_u_f64_e64 s[12:13], v[10:11], v[26:27]
-; GFX7-NEXT: v_min_f64 v[26:27], v[12:13], v[28:29]
+; GFX7-NEXT: v_min_f64 v[10:11], v[10:11], v[26:27]
; GFX7-NEXT: v_cmp_u_f64_e64 s[14:15], v[12:13], v[28:29]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v32, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e32 v1, v33, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v16, 0, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v3, v17, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v18, 0, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v19, v34, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v20, 0, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v21, v34, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v8, v22, 0, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v9, v23, v34, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v10, v24, 0, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v11, v25, v34, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v12, v26, 0, s[14:15]
-; GFX7-NEXT: v_cndmask_b32_e64 v13, v27, v34, s[14:15]
+; GFX7-NEXT: v_min_f64 v[12:13], v[12:13], v[28:29]
+; GFX7-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v9, v9, v16, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[12:13]
+; GFX7-NEXT: v_cndmask_b32_e64 v11, v11, v16, s[12:13]
+; GFX7-NEXT: v_cndmask_b32_e64 v12, v12, 0, s[14:15]
+; GFX7-NEXT: v_cndmask_b32_e64 v13, v13, v16, s[14:15]
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: v_min_f64 v[16:17], v[14:15], v[30:31]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[14:15], v[30:31]
-; GFX7-NEXT: v_cndmask_b32_e64 v14, v16, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e32 v15, v17, v34, vcc
+; GFX7-NEXT: v_cmp_u_f64_e64 s[16:17], v[14:15], v[30:31]
+; GFX7-NEXT: v_min_f64 v[14:15], v[14:15], v[30:31]
+; GFX7-NEXT: v_cndmask_b32_e64 v15, v15, v16, s[16:17]
+; GFX7-NEXT: v_cndmask_b32_e64 v14, v14, 0, s[16:17]
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: v_minimum_v8f64:
; GFX8: ; %bb.0:
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX8-NEXT: v_min_f64 v[32:33], v[0:1], v[16:17]
; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[16:17]
-; GFX8-NEXT: v_min_f64 v[16:17], v[2:3], v[18:19]
+; GFX8-NEXT: v_min_f64 v[0:1], v[0:1], v[16:17]
; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[18:19]
-; GFX8-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX8-NEXT: v_min_f64 v[18:19], v[4:5], v[20:21]
+; GFX8-NEXT: v_min_f64 v[2:3], v[2:3], v[18:19]
; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
-; GFX8-NEXT: v_min_f64 v[20:21], v[6:7], v[22:23]
+; GFX8-NEXT: v_min_f64 v[4:5], v[4:5], v[20:21]
; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[6:7], v[22:23]
-; GFX8-NEXT: v_min_f64 v[22:23], v[8:9], v[24:25]
+; GFX8-NEXT: v_min_f64 v[6:7], v[6:7], v[22:23]
; GFX8-NEXT: v_cmp_u_f64_e64 s[10:11], v[8:9], v[24:25]
-; GFX8-NEXT: v_min_f64 v[24:25], v[10:11], v[26:27]
+; GFX8-NEXT: v_min_f64 v[8:9], v[8:9], v[24:25]
; GFX8-NEXT: v_cmp_u_f64_e64 s[12:13], v[10:11], v[26:27]
-; GFX8-NEXT: v_min_f64 v[26:27], v[12:13], v[28:29]
+; GFX8-NEXT: v_min_f64 v[10:11], v[10:11], v[26:27]
; GFX8-NEXT: v_cmp_u_f64_e64 s[14:15], v[12:13], v[28:29]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v32, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v33, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v16, 0, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v17, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v18, 0, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v19, v34, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v20, 0, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v21, v34, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v22, 0, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v23, v34, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v24, 0, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v25, v34, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v12, v26, 0, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e64 v13, v27, v34, s[14:15]
+; GFX8-NEXT: v_min_f64 v[12:13], v[12:13], v[28:29]
+; GFX8-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v9, v9, v16, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[12:13]
+; GFX8-NEXT: v_cndmask_b32_e64 v11, v11, v16, s[12:13]
+; GFX8-NEXT: v_cndmask_b32_e64 v12, v12, 0, s[14:15]
+; GFX8-NEXT: v_cndmask_b32_e64 v13, v13, v16, s[14:15]
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_min_f64 v[16:17], v[14:15], v[30:31]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[14:15], v[30:31]
-; GFX8-NEXT: v_cndmask_b32_e64 v14, v16, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v15, v17, v34, vcc
+; GFX8-NEXT: v_cmp_u_f64_e64 s[16:17], v[14:15], v[30:31]
+; GFX8-NEXT: v_min_f64 v[14:15], v[14:15], v[30:31]
+; GFX8-NEXT: v_cndmask_b32_e64 v15, v15, v16, s[16:17]
+; GFX8-NEXT: v_cndmask_b32_e64 v14, v14, 0, s[16:17]
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX900-LABEL: v_minimum_v8f64:
; GFX900: ; %bb.0:
; GFX900-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX900-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX900-NEXT: v_min_f64 v[32:33], v[0:1], v[16:17]
; GFX900-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[16:17]
-; GFX900-NEXT: v_min_f64 v[16:17], v[2:3], v[18:19]
+; GFX900-NEXT: v_min_f64 v[0:1], v[0:1], v[16:17]
; GFX900-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[18:19]
-; GFX900-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX900-NEXT: v_min_f64 v[18:19], v[4:5], v[20:21]
+; GFX900-NEXT: v_min_f64 v[2:3], v[2:3], v[18:19]
; GFX900-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
-; GFX900-NEXT: v_min_f64 v[20:21], v[6:7], v[22:23]
+; GFX900-NEXT: v_min_f64 v[4:5], v[4:5], v[20:21]
; GFX900-NEXT: v_cmp_u_f64_e64 s[8:9], v[6:7], v[22:23]
-; GFX900-NEXT: v_min_f64 v[22:23], v[8:9], v[24:25]
+; GFX900-NEXT: v_min_f64 v[6:7], v[6:7], v[22:23]
; GFX900-NEXT: v_cmp_u_f64_e64 s[10:11], v[8:9], v[24:25]
-; GFX900-NEXT: v_min_f64 v[24:25], v[10:11], v[26:27]
+; GFX900-NEXT: v_min_f64 v[8:9], v[8:9], v[24:25]
; GFX900-NEXT: v_cmp_u_f64_e64 s[12:13], v[10:11], v[26:27]
-; GFX900-NEXT: v_min_f64 v[26:27], v[12:13], v[28:29]
+; GFX900-NEXT: v_min_f64 v[10:11], v[10:11], v[26:27]
; GFX900-NEXT: v_cmp_u_f64_e64 s[14:15], v[12:13], v[28:29]
-; GFX900-NEXT: v_cndmask_b32_e64 v0, v32, 0, vcc
-; GFX900-NEXT: v_cndmask_b32_e32 v1, v33, v34, vcc
-; GFX900-NEXT: v_cndmask_b32_e64 v2, v16, 0, s[4:5]
-; GFX900-NEXT: v_cndmask_b32_e64 v3, v17, v34, s[4:5]
-; GFX900-NEXT: v_cndmask_b32_e64 v4, v18, 0, s[6:7]
-; GFX900-NEXT: v_cndmask_b32_e64 v5, v19, v34, s[6:7]
-; GFX900-NEXT: v_cndmask_b32_e64 v6, v20, 0, s[8:9]
-; GFX900-NEXT: v_cndmask_b32_e64 v7, v21, v34, s[8:9]
-; GFX900-NEXT: v_cndmask_b32_e64 v8, v22, 0, s[10:11]
-; GFX900-NEXT: v_cndmask_b32_e64 v9, v23, v34, s[10:11]
-; GFX900-NEXT: v_cndmask_b32_e64 v10, v24, 0, s[12:13]
-; GFX900-NEXT: v_cndmask_b32_e64 v11, v25, v34, s[12:13]
-; GFX900-NEXT: v_cndmask_b32_e64 v12, v26, 0, s[14:15]
-; GFX900-NEXT: v_cndmask_b32_e64 v13, v27, v34, s[14:15]
+; GFX900-NEXT: v_min_f64 v[12:13], v[12:13], v[28:29]
+; GFX900-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX900-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX900-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX900-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
+; GFX900-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX900-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX900-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX900-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[8:9]
+; GFX900-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[8:9]
+; GFX900-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[10:11]
+; GFX900-NEXT: v_cndmask_b32_e64 v9, v9, v16, s[10:11]
+; GFX900-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[12:13]
+; GFX900-NEXT: v_cndmask_b32_e64 v11, v11, v16, s[12:13]
+; GFX900-NEXT: v_cndmask_b32_e64 v12, v12, 0, s[14:15]
+; GFX900-NEXT: v_cndmask_b32_e64 v13, v13, v16, s[14:15]
; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_min_f64 v[16:17], v[14:15], v[30:31]
-; GFX900-NEXT: v_cmp_u_f64_e32 vcc, v[14:15], v[30:31]
-; GFX900-NEXT: v_cndmask_b32_e64 v14, v16, 0, vcc
-; GFX900-NEXT: v_cndmask_b32_e32 v15, v17, v34, vcc
+; GFX900-NEXT: v_cmp_u_f64_e64 s[16:17], v[14:15], v[30:31]
+; GFX900-NEXT: v_min_f64 v[14:15], v[14:15], v[30:31]
+; GFX900-NEXT: v_cndmask_b32_e64 v15, v15, v16, s[16:17]
+; GFX900-NEXT: v_cndmask_b32_e64 v14, v14, 0, s[16:17]
; GFX900-NEXT: s_setpc_b64 s[30:31]
;
; GFX950-LABEL: v_minimum_v8f64:
diff --git a/llvm/test/CodeGen/AMDGPU/load-constant-i32.ll b/llvm/test/CodeGen/AMDGPU/load-constant-i32.ll
index 665f84001a5f5..af8e87a981373 100644
--- a/llvm/test/CodeGen/AMDGPU/load-constant-i32.ll
+++ b/llvm/test/CodeGen/AMDGPU/load-constant-i32.ll
@@ -5115,10 +5115,10 @@ define amdgpu_kernel void @constant_zextload_v32i32_to_v32i64(ptr addrspace(1) %
; GFX7-HSA-NEXT: v_mov_b32_e32 v8, s0
; GFX7-HSA-NEXT: s_add_u32 s0, s36, 0xc0
; GFX7-HSA-NEXT: s_addc_u32 s1, s37, 0
+; GFX7-HSA-NEXT: s_load_dwordx16 s[16:31], s[38:39], 0x10
; GFX7-HSA-NEXT: v_mov_b32_e32 v11, s1
; GFX7-HSA-NEXT: v_mov_b32_e32 v10, s0
; GFX7-HSA-NEXT: s_add_u32 s0, s36, 0xb0
-; GFX7-HSA-NEXT: s_load_dwordx16 s[16:31], s[38:39], 0x10
; GFX7-HSA-NEXT: s_addc_u32 s1, s37, 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v13, s1
; GFX7-HSA-NEXT: v_mov_b32_e32 v12, s0
@@ -5167,61 +5167,67 @@ define amdgpu_kernel void @constant_zextload_v32i32_to_v32i64(ptr addrspace(1) %
; GFX7-HSA-NEXT: s_addc_u32 s17, s37, 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s16
; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s17
-; GFX7-HSA-NEXT: s_add_u32 s16, s36, 0x70
-; GFX7-HSA-NEXT: s_addc_u32 s17, s37, 0
-; GFX7-HSA-NEXT: v_mov_b32_e32 v6, s16
-; GFX7-HSA-NEXT: v_mov_b32_e32 v7, s17
-; GFX7-HSA-NEXT: s_add_u32 s16, s36, 0x60
-; GFX7-HSA-NEXT: s_addc_u32 s17, s37, 0
-; GFX7-HSA-NEXT: v_mov_b32_e32 v8, s16
-; GFX7-HSA-NEXT: v_mov_b32_e32 v9, s17
-; GFX7-HSA-NEXT: s_add_u32 s16, s36, 0x50
-; GFX7-HSA-NEXT: s_addc_u32 s17, s37, 0
-; GFX7-HSA-NEXT: v_mov_b32_e32 v10, s16
-; GFX7-HSA-NEXT: v_mov_b32_e32 v11, s17
-; GFX7-HSA-NEXT: s_add_u32 s16, s36, 64
-; GFX7-HSA-NEXT: s_addc_u32 s17, s37, 0
-; GFX7-HSA-NEXT: v_mov_b32_e32 v12, s16
-; GFX7-HSA-NEXT: v_mov_b32_e32 v13, s17
-; GFX7-HSA-NEXT: s_add_u32 s16, s36, 48
; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
-; GFX7-HSA-NEXT: s_addc_u32 s17, s37, 0
; GFX7-HSA-NEXT: s_waitcnt lgkmcnt(0)
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s14
+; GFX7-HSA-NEXT: s_add_u32 s14, s36, 0x70
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s15
-; GFX7-HSA-NEXT: flat_store_dwordx4 v[6:7], v[0:3]
-; GFX7-HSA-NEXT: s_add_u32 s14, s36, 32
+; GFX7-HSA-NEXT: s_addc_u32 s15, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s14
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s15
+; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GFX7-HSA-NEXT: s_nop 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s12
+; GFX7-HSA-NEXT: s_add_u32 s12, s36, 0x60
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s13
-; GFX7-HSA-NEXT: flat_store_dwordx4 v[8:9], v[0:3]
-; GFX7-HSA-NEXT: s_addc_u32 s15, s37, 0
+; GFX7-HSA-NEXT: s_addc_u32 s13, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s12
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s13
+; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GFX7-HSA-NEXT: s_nop 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s10
+; GFX7-HSA-NEXT: s_add_u32 s10, s36, 0x50
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s11
-; GFX7-HSA-NEXT: flat_store_dwordx4 v[10:11], v[0:3]
-; GFX7-HSA-NEXT: v_mov_b32_e32 v14, s16
+; GFX7-HSA-NEXT: s_addc_u32 s11, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s10
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s11
+; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GFX7-HSA-NEXT: s_nop 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s8
+; GFX7-HSA-NEXT: s_add_u32 s8, s36, 64
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s9
-; GFX7-HSA-NEXT: v_mov_b32_e32 v15, s17
-; GFX7-HSA-NEXT: s_add_u32 s12, s36, 16
-; GFX7-HSA-NEXT: flat_store_dwordx4 v[12:13], v[0:3]
-; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s14
+; GFX7-HSA-NEXT: s_addc_u32 s9, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s8
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s9
+; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GFX7-HSA-NEXT: s_nop 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s6
+; GFX7-HSA-NEXT: s_add_u32 s6, s36, 48
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s7
-; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s15
-; GFX7-HSA-NEXT: s_addc_u32 s13, s37, 0
-; GFX7-HSA-NEXT: flat_store_dwordx4 v[14:15], v[0:3]
-; GFX7-HSA-NEXT: v_mov_b32_e32 v6, s12
+; GFX7-HSA-NEXT: s_addc_u32 s7, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s6
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s7
+; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GFX7-HSA-NEXT: s_nop 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s4
+; GFX7-HSA-NEXT: s_add_u32 s4, s36, 32
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s5
-; GFX7-HSA-NEXT: v_mov_b32_e32 v7, s13
+; GFX7-HSA-NEXT: s_addc_u32 s5, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s4
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s5
; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
-; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s36
+; GFX7-HSA-NEXT: s_nop 0
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s2
+; GFX7-HSA-NEXT: s_add_u32 s2, s36, 16
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s3
-; GFX7-HSA-NEXT: flat_store_dwordx4 v[6:7], v[0:3]
-; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s37
+; GFX7-HSA-NEXT: s_addc_u32 s3, s37, 0
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s3
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s2
+; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GFX7-HSA-NEXT: v_mov_b32_e32 v4, s36
; GFX7-HSA-NEXT: v_mov_b32_e32 v0, s0
; GFX7-HSA-NEXT: v_mov_b32_e32 v2, s1
+; GFX7-HSA-NEXT: v_mov_b32_e32 v5, s37
; GFX7-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
; GFX7-HSA-NEXT: s_endpgm
;
diff --git a/llvm/test/CodeGen/AMDGPU/load-global-i16.ll b/llvm/test/CodeGen/AMDGPU/load-global-i16.ll
index b150958ddd51d..6a9346b2da5dd 100644
--- a/llvm/test/CodeGen/AMDGPU/load-global-i16.ll
+++ b/llvm/test/CodeGen/AMDGPU/load-global-i16.ll
@@ -3431,116 +3431,102 @@ define amdgpu_kernel void @global_zextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-NOHSA-SI-NEXT: s_mov_b32 s8, s6
; GCN-NOHSA-SI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-SI-NEXT: s_mov_b32 s11, s3
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:16
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:32
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:48
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[27:30], off, s[8:11], 0 offset:64
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[31:34], off, s[8:11], 0 offset:80
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[35:38], off, s[8:11], 0 offset:96
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[39:42], off, s[8:11], 0 offset:112
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[16:19], off, s[8:11], 0
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[20:23], off, s[8:11], 0 offset:16
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[24:27], off, s[8:11], 0 offset:32
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0 offset:48
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:64
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[32:35], off, s[8:11], 0 offset:80
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[36:39], off, s[8:11], 0 offset:96
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[40:43], off, s[8:11], 0 offset:112
; GCN-NOHSA-SI-NEXT: s_addc_u32 s13, s13, 0
; GCN-NOHSA-SI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-SI-NEXT: s_mov_b32 s1, s5
; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(7)
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v18, 16, v15
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v16, 16, v14
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v17, 0xffff, v15
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v15, 0xffff, v14
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v15, off, s[12:15], 0 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v16, off, s[12:15], 0 offset:4 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v17, off, s[12:15], 0 offset:8 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v18, off, s[12:15], 0 offset:12 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v21, 16, v13
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v19, 16, v12
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v20, 0xffff, v13
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(0)
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v18, 0xffff, v12
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v18, off, s[12:15], 0 offset:16 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v19, off, s[12:15], 0 offset:20 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v20, off, s[12:15], 0 offset:24 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v21, off, s[12:15], 0 offset:28 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(14)
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v26, 16, v11
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v24, 16, v10
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(2)
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v19, 16, v9
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v17, 16, v8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v25, 0xffff, v11
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v23, 0xffff, v10
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v18, 0xffff, v9
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v16, 0xffff, v8
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(12)
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v11, 16, v3
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v9, 16, v2
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v50, 16, v1
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v48, 16, v0
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v10, 0xffff, v3
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v8, 0xffff, v2
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v49, 0xffff, v1
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v47, 0xffff, v0
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(8)
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v3, 16, v40
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v1, 16, v39
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v2, 0xffff, v40
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v0, 0xffff, v39
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v15, 16, v7
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v13, 16, v6
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v46, 16, v5
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v44, 16, v4
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v14, 0xffff, v7
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v12, 0xffff, v6
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v45, 0xffff, v5
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v43, 0xffff, v4
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v54, 16, v30
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v52, 16, v29
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v58, 16, v28
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v56, 16, v27
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v53, 0xffff, v30
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v51, 0xffff, v29
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v57, 0xffff, v28
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v55, 0xffff, v27
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v30, 16, v34
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v28, 16, v33
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v62, 16, v32
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v60, 16, v31
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v29, 0xffff, v34
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v27, 0xffff, v33
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v61, 0xffff, v32
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v59, 0xffff, v31
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v34, 16, v38
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v32, 16, v37
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v7, 16, v36
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v5, 16, v35
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v33, 0xffff, v38
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v31, 0xffff, v37
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v6, 0xffff, v36
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v4, 0xffff, v35
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v38, 16, v42
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v36, 16, v41
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v37, 0xffff, v42
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v35, 0xffff, v41
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v3, 16, v19
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v1, 16, v18
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v2, 0xffff, v19
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v0, 0xffff, v18
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v0, off, s[12:15], 0 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v2, off, s[12:15], 0 offset:8 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v3, off, s[12:15], 0 offset:12 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v7, 16, v17
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v5, 16, v16
+; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(4) expcnt(0)
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v3, 16, v41
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v1, 16, v40
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v2, 0xffff, v41
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v0, 0xffff, v40
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v11, 16, v23
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v9, 16, v22
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v6, 0xffff, v17
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v4, 0xffff, v16
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v19, 16, v21
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v17, 16, v20
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v10, 0xffff, v23
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v8, 0xffff, v22
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v18, 0xffff, v21
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v16, 0xffff, v20
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v23, 16, v27
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v21, 16, v26
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v47, 16, v25
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v45, 16, v24
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v22, 0xffff, v27
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v20, 0xffff, v26
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v46, 0xffff, v25
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v44, 0xffff, v24
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v27, 16, v15
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v25, 16, v14
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v51, 16, v13
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v49, 16, v12
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v26, 0xffff, v15
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v24, 0xffff, v14
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v50, 0xffff, v13
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v48, 0xffff, v12
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v15, 16, v31
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v13, 16, v30
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v55, 16, v29
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v53, 16, v28
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v14, 0xffff, v31
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v12, 0xffff, v30
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v54, 0xffff, v29
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v52, 0xffff, v28
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v31, 16, v35
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v29, 16, v34
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v59, 16, v33
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v57, 16, v32
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v30, 0xffff, v35
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v28, 0xffff, v34
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v58, 0xffff, v33
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v56, 0xffff, v32
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v35, 16, v39
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v33, 16, v38
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v63, 16, v37
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v61, 16, v36
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v34, 0xffff, v39
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v32, 0xffff, v38
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v62, 0xffff, v37
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v60, 0xffff, v36
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v39, 16, v43
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v37, 16, v42
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v38, 0xffff, v43
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v36, 0xffff, v42
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[35:38], off, s[0:3], 0 offset:240
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:192
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[31:34], off, s[0:3], 0 offset:208
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[59:62], off, s[0:3], 0 offset:160
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[27:30], off, s[0:3], 0 offset:176
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[55:58], off, s[0:3], 0 offset:128
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[51:54], off, s[0:3], 0 offset:144
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[47:50], off, s[0:3], 0 offset:96
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:112
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[43:46], off, s[0:3], 0 offset:64
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:80
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[36:39], off, s[0:3], 0 offset:240
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[60:63], off, s[0:3], 0 offset:192
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:208
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[56:59], off, s[0:3], 0 offset:160
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:176
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[52:55], off, s[0:3], 0 offset:128
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:144
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[48:51], off, s[0:3], 0 offset:96
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:112
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[44:47], off, s[0:3], 0 offset:64
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:80
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:32
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[23:26], off, s[0:3], 0 offset:48
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:16 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:20 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:24 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:28 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(0)
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(0)
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0
; GCN-NOHSA-SI-NEXT: buffer_load_dword v0, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-NOHSA-SI-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-NOHSA-SI-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
@@ -3558,187 +3544,186 @@ define amdgpu_kernel void @global_zextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-HSA-NEXT: s_waitcnt lgkmcnt(0)
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
-; GCN-HSA-NEXT: flat_load_dwordx4 v[24:27], v[0:1]
-; GCN-HSA-NEXT: s_add_u32 s4, s2, 16
-; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
-; GCN-HSA-NEXT: s_add_u32 s6, s2, 32
+; GCN-HSA-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
+; GCN-HSA-NEXT: s_add_u32 s6, s2, 16
; GCN-HSA-NEXT: s_addc_u32 s7, s3, 0
+; GCN-HSA-NEXT: s_add_u32 s4, s2, 32
+; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
; GCN-HSA-NEXT: s_add_u32 s8, s2, 48
; GCN-HSA-NEXT: s_addc_u32 s9, s3, 0
; GCN-HSA-NEXT: s_add_u32 s10, s2, 64
; GCN-HSA-NEXT: s_addc_u32 s11, s3, 0
-; GCN-HSA-NEXT: v_mov_b32_e32 v17, s9
-; GCN-HSA-NEXT: v_mov_b32_e32 v16, s8
-; GCN-HSA-NEXT: flat_load_dwordx4 v[20:23], v[16:17]
+; GCN-HSA-NEXT: v_mov_b32_e32 v19, s9
+; GCN-HSA-NEXT: v_mov_b32_e32 v18, s8
+; GCN-HSA-NEXT: flat_load_dwordx4 v[24:27], v[18:19]
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s10
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s11
; GCN-HSA-NEXT: s_add_u32 s10, s2, 0x50
; GCN-HSA-NEXT: s_addc_u32 s11, s3, 0
+; GCN-HSA-NEXT: flat_load_dwordx4 v[0:3], v[0:1]
; GCN-HSA-NEXT: v_mov_b32_e32 v4, s10
; GCN-HSA-NEXT: v_mov_b32_e32 v5, s11
; GCN-HSA-NEXT: s_add_u32 s10, s2, 0x60
-; GCN-HSA-NEXT: flat_load_dwordx4 v[0:3], v[0:1]
; GCN-HSA-NEXT: flat_load_dwordx4 v[8:11], v[4:5]
; GCN-HSA-NEXT: s_addc_u32 s11, s3, 0
-; GCN-HSA-NEXT: s_add_u32 s2, s2, 0x70
; GCN-HSA-NEXT: v_mov_b32_e32 v4, s10
; GCN-HSA-NEXT: v_mov_b32_e32 v5, s11
-; GCN-HSA-NEXT: s_addc_u32 s3, s3, 0
+; GCN-HSA-NEXT: s_add_u32 s2, s2, 0x70
; GCN-HSA-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GCN-HSA-NEXT: v_mov_b32_e32 v13, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v12, s2
-; GCN-HSA-NEXT: flat_load_dwordx4 v[28:31], v[12:13]
-; GCN-HSA-NEXT: v_mov_b32_e32 v13, s5
-; GCN-HSA-NEXT: v_mov_b32_e32 v12, s4
-; GCN-HSA-NEXT: v_mov_b32_e32 v15, s7
-; GCN-HSA-NEXT: v_mov_b32_e32 v14, s6
-; GCN-HSA-NEXT: flat_load_dwordx4 v[16:19], v[12:13]
-; GCN-HSA-NEXT: flat_load_dwordx4 v[12:15], v[14:15]
+; GCN-HSA-NEXT: s_addc_u32 s3, s3, 0
+; GCN-HSA-NEXT: v_mov_b32_e32 v17, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v16, s2
+; GCN-HSA-NEXT: flat_load_dwordx4 v[28:31], v[16:17]
+; GCN-HSA-NEXT: v_mov_b32_e32 v17, s7
+; GCN-HSA-NEXT: v_mov_b32_e32 v16, s6
; GCN-HSA-NEXT: s_add_u32 s2, s0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCN-HSA-NEXT: v_mov_b32_e32 v37, s1
; GCN-HSA-NEXT: v_mov_b32_e32 v36, s0
-; GCN-HSA-NEXT: s_waitcnt vmcnt(7)
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v35, 16, v25
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v33, 16, v24
-; GCN-HSA-NEXT: v_and_b32_e32 v34, 0xffff, v25
-; GCN-HSA-NEXT: v_and_b32_e32 v32, 0xffff, v24
-; GCN-HSA-NEXT: v_mov_b32_e32 v25, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v24, s2
+; GCN-HSA-NEXT: s_waitcnt vmcnt(5)
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v35, 16, v13
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v33, 16, v12
+; GCN-HSA-NEXT: v_and_b32_e32 v34, 0xffff, v13
+; GCN-HSA-NEXT: v_and_b32_e32 v32, 0xffff, v12
+; GCN-HSA-NEXT: v_mov_b32_e32 v13, s5
+; GCN-HSA-NEXT: v_mov_b32_e32 v12, s4
+; GCN-HSA-NEXT: flat_load_dwordx4 v[20:23], v[16:17]
+; GCN-HSA-NEXT: flat_load_dwordx4 v[16:19], v[12:13]
+; GCN-HSA-NEXT: v_mov_b32_e32 v13, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v12, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xe0
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCN-HSA-NEXT: s_add_u32 s4, s0, 0xf0
; GCN-HSA-NEXT: s_addc_u32 s5, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[36:37], v[32:35]
-; GCN-HSA-NEXT: v_mov_b32_e32 v37, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v36, s2
-; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xc0
-; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: s_add_u32 s6, s0, 0xd0
+; GCN-HSA-NEXT: s_add_u32 s6, s0, 0xc0
; GCN-HSA-NEXT: s_addc_u32 s7, s1, 0
-; GCN-HSA-NEXT: s_add_u32 s8, s0, 0xa0
+; GCN-HSA-NEXT: s_add_u32 s8, s0, 0xd0
; GCN-HSA-NEXT: s_addc_u32 s9, s1, 0
-; GCN-HSA-NEXT: s_add_u32 s10, s0, 0xb0
+; GCN-HSA-NEXT: s_add_u32 s10, s0, 0xa0
; GCN-HSA-NEXT: s_addc_u32 s11, s1, 0
-; GCN-HSA-NEXT: s_add_u32 s12, s0, 0x80
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v35, 16, v27
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v33, 16, v26
-; GCN-HSA-NEXT: v_and_b32_e32 v34, 0xffff, v27
-; GCN-HSA-NEXT: v_and_b32_e32 v32, 0xffff, v26
+; GCN-HSA-NEXT: s_add_u32 s12, s0, 0xb0
; GCN-HSA-NEXT: s_addc_u32 s13, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[24:25], v[32:35]
+; GCN-HSA-NEXT: s_add_u32 s14, s0, 0x80
+; GCN-HSA-NEXT: flat_store_dwordx4 v[36:37], v[32:35]
+; GCN-HSA-NEXT: s_addc_u32 s15, s1, 0
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v35, 16, v15
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v33, 16, v14
+; GCN-HSA-NEXT: v_and_b32_e32 v34, 0xffff, v15
+; GCN-HSA-NEXT: v_and_b32_e32 v32, 0xffff, v14
+; GCN-HSA-NEXT: flat_store_dwordx4 v[12:13], v[32:35]
; GCN-HSA-NEXT: s_waitcnt vmcnt(7)
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v27, 16, v1
-; GCN-HSA-NEXT: v_mov_b32_e32 v33, s13
-; GCN-HSA-NEXT: v_mov_b32_e32 v32, s12
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v25, 16, v0
-; GCN-HSA-NEXT: v_and_b32_e32 v26, 0xffff, v1
-; GCN-HSA-NEXT: v_and_b32_e32 v24, 0xffff, v0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s8
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s9
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v15, 16, v1
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v13, 16, v0
+; GCN-HSA-NEXT: v_and_b32_e32 v14, 0xffff, v1
+; GCN-HSA-NEXT: v_and_b32_e32 v12, 0xffff, v0
+; GCN-HSA-NEXT: v_mov_b32_e32 v0, s14
+; GCN-HSA-NEXT: v_mov_b32_e32 v1, s15
+; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GCN-HSA-NEXT: v_mov_b32_e32 v0, s10
; GCN-HSA-NEXT: s_waitcnt vmcnt(7)
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v27, 16, v9
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v25, 16, v8
-; GCN-HSA-NEXT: v_and_b32_e32 v26, 0xffff, v9
-; GCN-HSA-NEXT: v_and_b32_e32 v24, 0xffff, v8
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
-; GCN-HSA-NEXT: v_mov_b32_e32 v8, s10
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v27, 16, v11
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v25, 16, v10
-; GCN-HSA-NEXT: v_and_b32_e32 v26, 0xffff, v11
-; GCN-HSA-NEXT: v_and_b32_e32 v24, 0xffff, v10
-; GCN-HSA-NEXT: v_mov_b32_e32 v9, s11
-; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[24:27]
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v15, 16, v9
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v13, 16, v8
+; GCN-HSA-NEXT: v_and_b32_e32 v14, 0xffff, v9
+; GCN-HSA-NEXT: v_and_b32_e32 v12, 0xffff, v8
+; GCN-HSA-NEXT: v_mov_b32_e32 v1, s11
+; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
+; GCN-HSA-NEXT: v_mov_b32_e32 v8, s12
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v14, 16, v11
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v12, 16, v10
+; GCN-HSA-NEXT: v_and_b32_e32 v13, 0xffff, v11
+; GCN-HSA-NEXT: v_and_b32_e32 v11, 0xffff, v10
+; GCN-HSA-NEXT: v_mov_b32_e32 v9, s13
+; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[11:14]
; GCN-HSA-NEXT: s_waitcnt vmcnt(8)
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v11, 16, v5
; GCN-HSA-NEXT: v_lshrrev_b32_e32 v9, 16, v4
+; GCN-HSA-NEXT: v_mov_b32_e32 v13, s7
+; GCN-HSA-NEXT: v_mov_b32_e32 v12, s6
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v11, 16, v5
; GCN-HSA-NEXT: v_and_b32_e32 v10, 0xffff, v5
; GCN-HSA-NEXT: v_and_b32_e32 v8, 0xffff, v4
-; GCN-HSA-NEXT: v_mov_b32_e32 v5, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v4, s2
+; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
+; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
+; GCN-HSA-NEXT: flat_store_dwordx4 v[12:13], v[8:11]
+; GCN-HSA-NEXT: v_mov_b32_e32 v4, s8
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v10, 16, v7
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v8, 16, v6
+; GCN-HSA-NEXT: v_and_b32_e32 v9, 0xffff, v7
+; GCN-HSA-NEXT: v_and_b32_e32 v7, 0xffff, v6
+; GCN-HSA-NEXT: v_mov_b32_e32 v5, s9
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x90
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s4
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s5
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v27, 16, v7
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v25, 16, v6
-; GCN-HSA-NEXT: v_and_b32_e32 v26, 0xffff, v7
-; GCN-HSA-NEXT: v_and_b32_e32 v24, 0xffff, v6
-; GCN-HSA-NEXT: v_mov_b32_e32 v33, s7
-; GCN-HSA-NEXT: v_mov_b32_e32 v32, s6
-; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[8:11]
-; GCN-HSA-NEXT: s_waitcnt vmcnt(8)
-; GCN-HSA-NEXT: v_and_b32_e32 v7, 0xffff, v28
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v10, 16, v29
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v8, 16, v28
-; GCN-HSA-NEXT: v_and_b32_e32 v9, 0xffff, v29
+; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[7:10]
+; GCN-HSA-NEXT: s_waitcnt vmcnt(9)
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v5, 16, v28
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v7, 16, v29
+; GCN-HSA-NEXT: v_and_b32_e32 v6, 0xffff, v29
+; GCN-HSA-NEXT: v_and_b32_e32 v4, 0xffff, v28
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v6, 16, v3
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v27, 16, v31
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v25, 16, v30
-; GCN-HSA-NEXT: v_and_b32_e32 v26, 0xffff, v31
-; GCN-HSA-NEXT: v_and_b32_e32 v24, 0xffff, v30
-; GCN-HSA-NEXT: flat_store_dwordx4 v[36:37], v[7:10]
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[24:27]
+; GCN-HSA-NEXT: v_mov_b32_e32 v14, s5
+; GCN-HSA-NEXT: v_mov_b32_e32 v13, s4
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v11, 16, v31
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v9, 16, v30
+; GCN-HSA-NEXT: v_and_b32_e32 v10, 0xffff, v31
+; GCN-HSA-NEXT: v_and_b32_e32 v8, 0xffff, v30
+; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
+; GCN-HSA-NEXT: flat_store_dwordx4 v[13:14], v[8:11]
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x60
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v4, 16, v2
-; GCN-HSA-NEXT: v_and_b32_e32 v5, 0xffff, v3
-; GCN-HSA-NEXT: v_and_b32_e32 v3, 0xffff, v2
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[3:6]
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v1, 16, v20
; GCN-HSA-NEXT: v_mov_b32_e32 v5, s3
; GCN-HSA-NEXT: v_mov_b32_e32 v4, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x70
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v12, 16, v3
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v10, 16, v2
+; GCN-HSA-NEXT: v_and_b32_e32 v11, 0xffff, v3
+; GCN-HSA-NEXT: v_and_b32_e32 v9, 0xffff, v2
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v3, 16, v21
-; GCN-HSA-NEXT: v_and_b32_e32 v2, 0xffff, v21
-; GCN-HSA-NEXT: v_and_b32_e32 v0, 0xffff, v20
-; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
+; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[9:12]
; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
+; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
; GCN-HSA-NEXT: s_add_u32 s2, s0, 64
-; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v7, 16, v23
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v5, 16, v22
-; GCN-HSA-NEXT: v_and_b32_e32 v6, 0xffff, v23
-; GCN-HSA-NEXT: v_and_b32_e32 v4, 0xffff, v22
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: s_waitcnt vmcnt(12)
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v1, 16, v18
-; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
-; GCN-HSA-NEXT: s_waitcnt vmcnt(12)
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v11, 16, v15
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v7, 16, v17
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v5, 16, v16
-; GCN-HSA-NEXT: v_and_b32_e32 v0, 0xffff, v18
-; GCN-HSA-NEXT: v_and_b32_e32 v6, 0xffff, v17
-; GCN-HSA-NEXT: v_and_b32_e32 v4, 0xffff, v16
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v18, 16, v13
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v16, 16, v12
-; GCN-HSA-NEXT: v_and_b32_e32 v10, 0xffff, v15
-; GCN-HSA-NEXT: v_and_b32_e32 v17, 0xffff, v13
-; GCN-HSA-NEXT: v_and_b32_e32 v15, 0xffff, v12
-; GCN-HSA-NEXT: v_mov_b32_e32 v13, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v12, s2
+; GCN-HSA-NEXT: s_waitcnt vmcnt(10)
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v15, 16, v17
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v13, 16, v16
+; GCN-HSA-NEXT: v_and_b32_e32 v14, 0xffff, v17
+; GCN-HSA-NEXT: v_and_b32_e32 v12, 0xffff, v16
+; GCN-HSA-NEXT: v_mov_b32_e32 v17, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v16, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x50
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[12:13], v[15:18]
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v3, 16, v25
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v1, 16, v24
+; GCN-HSA-NEXT: v_and_b32_e32 v2, 0xffff, v25
+; GCN-HSA-NEXT: v_and_b32_e32 v0, 0xffff, v24
+; GCN-HSA-NEXT: flat_store_dwordx4 v[16:17], v[12:15]
+; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
; GCN-HSA-NEXT: v_mov_b32_e32 v13, s3
; GCN-HSA-NEXT: v_mov_b32_e32 v12, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 32
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v7, 16, v27
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v5, 16, v26
+; GCN-HSA-NEXT: v_and_b32_e32 v6, 0xffff, v27
+; GCN-HSA-NEXT: v_and_b32_e32 v4, 0xffff, v26
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v9, 16, v14
-; GCN-HSA-NEXT: v_and_b32_e32 v8, 0xffff, v14
+; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v11, 16, v19
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v9, 16, v18
+; GCN-HSA-NEXT: v_and_b32_e32 v10, 0xffff, v19
+; GCN-HSA-NEXT: v_and_b32_e32 v8, 0xffff, v18
; GCN-HSA-NEXT: s_add_u32 s0, s0, 48
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v7, 16, v21
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v5, 16, v20
+; GCN-HSA-NEXT: v_and_b32_e32 v6, 0xffff, v21
+; GCN-HSA-NEXT: v_and_b32_e32 v4, 0xffff, v20
; GCN-HSA-NEXT: flat_store_dwordx4 v[12:13], v[8:11]
; GCN-HSA-NEXT: s_addc_u32 s1, s1, 0
; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
-; GCN-HSA-NEXT: v_lshrrev_b32_e32 v3, 16, v19
-; GCN-HSA-NEXT: v_and_b32_e32 v2, 0xffff, v19
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v3, 16, v23
+; GCN-HSA-NEXT: v_lshrrev_b32_e32 v1, 16, v22
+; GCN-HSA-NEXT: v_and_b32_e32 v2, 0xffff, v23
+; GCN-HSA-NEXT: v_and_b32_e32 v0, 0xffff, v22
; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
; GCN-HSA-NEXT: s_nop 0
; GCN-HSA-NEXT: v_mov_b32_e32 v5, s1
@@ -3761,113 +3746,102 @@ define amdgpu_kernel void @global_zextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-NOHSA-VI-NEXT: s_mov_b32 s8, s6
; GCN-NOHSA-VI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-VI-NEXT: s_mov_b32 s11, s3
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:16
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:32
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:48
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[27:30], off, s[8:11], 0 offset:64
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[31:34], off, s[8:11], 0 offset:80
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[35:38], off, s[8:11], 0 offset:96
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[39:42], off, s[8:11], 0 offset:112
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[16:19], off, s[8:11], 0
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[20:23], off, s[8:11], 0 offset:16
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[24:27], off, s[8:11], 0 offset:32
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0 offset:48
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:64
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[32:35], off, s[8:11], 0 offset:80
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[36:39], off, s[8:11], 0 offset:96
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[40:43], off, s[8:11], 0 offset:112
; GCN-NOHSA-VI-NEXT: s_addc_u32 s89, s89, 0
; GCN-NOHSA-VI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-VI-NEXT: s_mov_b32 s1, s5
; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(7)
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v18, 16, v15
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v16, 16, v14
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v17, 0xffff, v15
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v15, 0xffff, v14
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v15, off, s[88:91], 0 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v16, off, s[88:91], 0 offset:4 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v17, off, s[88:91], 0 offset:8 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v18, off, s[88:91], 0 offset:12 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v21, 16, v13
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v19, 16, v12
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v20, 0xffff, v13
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v18, 0xffff, v12
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v18, off, s[88:91], 0 offset:16 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v19, off, s[88:91], 0 offset:20 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v20, off, s[88:91], 0 offset:24 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v21, off, s[88:91], 0 offset:28 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(14)
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v26, 16, v11
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v24, 16, v10
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v19, 16, v9
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v17, 16, v8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v25, 0xffff, v11
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v23, 0xffff, v10
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v18, 0xffff, v9
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v16, 0xffff, v8
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(12)
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v11, 16, v3
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v9, 16, v2
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v50, 16, v1
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v48, 16, v0
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v10, 0xffff, v3
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v8, 0xffff, v2
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v49, 0xffff, v1
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v47, 0xffff, v0
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(8)
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v3, 16, v40
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v1, 16, v39
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v2, 0xffff, v40
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v0, 0xffff, v39
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v15, 16, v7
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v13, 16, v6
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v46, 16, v5
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v44, 16, v4
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v14, 0xffff, v7
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v12, 0xffff, v6
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v45, 0xffff, v5
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v43, 0xffff, v4
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v54, 16, v30
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v52, 16, v29
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v58, 16, v28
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v56, 16, v27
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v53, 0xffff, v30
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v51, 0xffff, v29
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v57, 0xffff, v28
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v55, 0xffff, v27
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v30, 16, v34
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v28, 16, v33
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v62, 16, v32
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v60, 16, v31
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v29, 0xffff, v34
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v27, 0xffff, v33
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v61, 0xffff, v32
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v59, 0xffff, v31
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v34, 16, v38
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v32, 16, v37
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v7, 16, v36
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v5, 16, v35
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v33, 0xffff, v38
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v31, 0xffff, v37
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v6, 0xffff, v36
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v4, 0xffff, v35
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v38, 16, v42
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v36, 16, v41
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v37, 0xffff, v42
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v35, 0xffff, v41
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v3, 16, v19
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v1, 16, v18
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v2, 0xffff, v19
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v0, 0xffff, v18
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v0, off, s[88:91], 0 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v1, off, s[88:91], 0 offset:4 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v2, off, s[88:91], 0 offset:8 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v3, off, s[88:91], 0 offset:12 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v7, 16, v17
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v5, 16, v16
+; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(4)
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v3, 16, v41
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v1, 16, v40
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v2, 0xffff, v41
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v0, 0xffff, v40
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v11, 16, v23
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v9, 16, v22
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v6, 0xffff, v17
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v4, 0xffff, v16
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v19, 16, v21
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v17, 16, v20
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v10, 0xffff, v23
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v8, 0xffff, v22
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v18, 0xffff, v21
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v16, 0xffff, v20
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v23, 16, v27
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v21, 16, v26
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v47, 16, v25
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v45, 16, v24
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v22, 0xffff, v27
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v20, 0xffff, v26
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v46, 0xffff, v25
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v44, 0xffff, v24
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v27, 16, v15
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v25, 16, v14
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v51, 16, v13
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v49, 16, v12
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v26, 0xffff, v15
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v24, 0xffff, v14
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v50, 0xffff, v13
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v48, 0xffff, v12
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v15, 16, v31
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v13, 16, v30
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v55, 16, v29
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v53, 16, v28
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v14, 0xffff, v31
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v12, 0xffff, v30
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v54, 0xffff, v29
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v52, 0xffff, v28
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v31, 16, v35
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v29, 16, v34
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v59, 16, v33
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v57, 16, v32
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v30, 0xffff, v35
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v28, 0xffff, v34
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v58, 0xffff, v33
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v56, 0xffff, v32
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v35, 16, v39
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v33, 16, v38
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v63, 16, v37
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v61, 16, v36
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v34, 0xffff, v39
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v32, 0xffff, v38
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v62, 0xffff, v37
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v60, 0xffff, v36
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v39, 16, v43
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v37, 16, v42
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v38, 0xffff, v43
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v36, 0xffff, v42
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[35:38], off, s[0:3], 0 offset:240
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:192
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[31:34], off, s[0:3], 0 offset:208
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[59:62], off, s[0:3], 0 offset:160
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[27:30], off, s[0:3], 0 offset:176
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[55:58], off, s[0:3], 0 offset:128
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[51:54], off, s[0:3], 0 offset:144
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[47:50], off, s[0:3], 0 offset:96
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:112
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[43:46], off, s[0:3], 0 offset:64
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:80
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[36:39], off, s[0:3], 0 offset:240
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[60:63], off, s[0:3], 0 offset:192
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:208
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[56:59], off, s[0:3], 0 offset:160
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:176
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[52:55], off, s[0:3], 0 offset:128
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:144
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[48:51], off, s[0:3], 0 offset:96
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:112
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[44:47], off, s[0:3], 0 offset:64
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:80
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:32
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[23:26], off, s[0:3], 0 offset:48
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v0, off, s[88:91], 0 offset:16 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v1, off, s[88:91], 0 offset:20 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v2, off, s[88:91], 0 offset:24 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v3, off, s[88:91], 0 offset:28 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(0)
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0
; GCN-NOHSA-VI-NEXT: buffer_load_dword v0, off, s[88:91], 0 ; 4-byte Folded Reload
; GCN-NOHSA-VI-NEXT: buffer_load_dword v1, off, s[88:91], 0 offset:4 ; 4-byte Folded Reload
; GCN-NOHSA-VI-NEXT: buffer_load_dword v2, off, s[88:91], 0 offset:8 ; 4-byte Folded Reload
@@ -4209,118 +4183,102 @@ define amdgpu_kernel void @global_sextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-NOHSA-SI-NEXT: s_mov_b32 s8, s6
; GCN-NOHSA-SI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-SI-NEXT: s_mov_b32 s11, s3
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:16
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:32
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:48
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[23:26], off, s[8:11], 0 offset:64
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[27:30], off, s[8:11], 0 offset:80
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[31:34], off, s[8:11], 0 offset:96
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[35:38], off, s[8:11], 0 offset:112
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0 offset:16
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[16:19], off, s[8:11], 0 offset:32
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[20:23], off, s[8:11], 0 offset:48
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[24:27], off, s[8:11], 0 offset:64
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:80
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[32:35], off, s[8:11], 0 offset:96
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[36:39], off, s[8:11], 0 offset:112
; GCN-NOHSA-SI-NEXT: s_addc_u32 s13, s13, 0
; GCN-NOHSA-SI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-SI-NEXT: s_mov_b32 s1, s5
; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(7)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v18, 16, v15
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v16, 16, v14
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v17, v15, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v15, v14, 0, 16
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v15, off, s[12:15], 0 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v16, off, s[12:15], 0 offset:4 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v17, off, s[12:15], 0 offset:8 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v18, off, s[12:15], 0 offset:12 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(2)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v16, 16, v13
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v14, 16, v12
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v15, v13, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v13, v12, 0, 16
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v13, off, s[12:15], 0 offset:16 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v14, off, s[12:15], 0 offset:20 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v15, off, s[12:15], 0 offset:24 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: buffer_store_dword v16, off, s[12:15], 0 offset:28 ; 4-byte Folded Spill
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(14)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v19, 16, v11
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(5)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v17, 16, v10
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(4)
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v18, v11, 0, 16
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(0)
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v16, v10, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v42, 16, v9
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v40, 16, v8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v41, v9, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v39, v8, 0, 16
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(12)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v11, 16, v3
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v9, 16, v2
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v10, v3, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v8, v2, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v50, 16, v1
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v48, 16, v0
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v49, v1, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v47, v0, 0, 16
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(8)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v3, 16, v36
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v1, 16, v35
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v2, v36, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v0, v35, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v15, 16, v7
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v13, 16, v6
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v14, v7, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v12, v6, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v46, 16, v5
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v44, 16, v4
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v45, v5, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v43, v4, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v54, 16, v26
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v52, 16, v25
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v53, v26, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v51, v25, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v58, 16, v24
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v56, 16, v23
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v57, v24, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v55, v23, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v26, 16, v30
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v24, 16, v29
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v25, v30, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v23, v29, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v62, 16, v28
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v60, 16, v27
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v61, v28, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v59, v27, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v30, 16, v34
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v28, 16, v33
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v29, v34, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v27, v33, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v7, 16, v32
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v5, 16, v31
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v6, v32, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v4, v31, 0, 16
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v34, 16, v38
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v32, 16, v37
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v33, v38, 0, 16
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v31, v37, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v3, 16, v11
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v1, 16, v10
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v2, v11, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v0, v10, 0, 16
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v0, off, s[12:15], 0 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v2, off, s[12:15], 0 offset:8 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: buffer_store_dword v3, off, s[12:15], 0 offset:12 ; 4-byte Folded Spill
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v7, 16, v9
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v5, 16, v8
+; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(4) expcnt(0)
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v3, 16, v37
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v1, 16, v36
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v2, v37, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v0, v36, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v6, v9, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v4, v8, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v11, 16, v15
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v9, 16, v14
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v10, v15, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v8, v14, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v43, 16, v13
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v41, 16, v12
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v42, v13, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v40, v12, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v15, 16, v19
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v13, 16, v18
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v14, v19, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v12, v18, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v47, 16, v17
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v45, 16, v16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v46, v17, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v44, v16, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v19, 16, v23
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v17, 16, v22
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v18, v23, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v16, v22, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v51, 16, v21
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v49, 16, v20
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v50, v21, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v48, v20, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v23, 16, v27
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v21, 16, v26
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v22, v27, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v20, v26, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v55, 16, v25
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v53, 16, v24
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v54, v25, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v52, v24, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v27, 16, v31
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v25, 16, v30
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v26, v31, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v24, v30, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v59, 16, v29
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v57, 16, v28
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v58, v29, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v56, v28, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v31, 16, v35
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v29, 16, v34
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v30, v35, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v28, v34, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v63, 16, v33
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v61, 16, v32
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v62, v33, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v60, v32, 0, 16
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v35, 16, v39
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v33, 16, v38
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v34, v39, 0, 16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v32, v38, 0, 16
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[31:34], off, s[0:3], 0 offset:240
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:192
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[27:30], off, s[0:3], 0 offset:208
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[59:62], off, s[0:3], 0 offset:160
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[23:26], off, s[0:3], 0 offset:176
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[55:58], off, s[0:3], 0 offset:128
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[51:54], off, s[0:3], 0 offset:144
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[47:50], off, s[0:3], 0 offset:96
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:112
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[43:46], off, s[0:3], 0 offset:64
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:240
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[60:63], off, s[0:3], 0 offset:192
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:208
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[56:59], off, s[0:3], 0 offset:160
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:176
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[52:55], off, s[0:3], 0 offset:128
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:144
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[48:51], off, s[0:3], 0 offset:96
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:112
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[44:47], off, s[0:3], 0 offset:64
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:80
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[39:42], off, s[0:3], 0 offset:32
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:48
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:16 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:20 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:24 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:28 ; 4-byte Folded Reload
-; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(0)
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0
-; GCN-NOHSA-SI-NEXT: s_waitcnt expcnt(0)
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[40:43], off, s[0:3], 0 offset:32
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0
; GCN-NOHSA-SI-NEXT: buffer_load_dword v0, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-NOHSA-SI-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-NOHSA-SI-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
@@ -4333,46 +4291,46 @@ define amdgpu_kernel void @global_sextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-HSA: ; %bb.0:
; GCN-HSA-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0
; GCN-HSA-NEXT: s_add_i32 s12, s12, s17
-; GCN-HSA-NEXT: s_mov_b32 flat_scratch_lo, s13
; GCN-HSA-NEXT: s_lshr_b32 flat_scratch_hi, s12, 8
+; GCN-HSA-NEXT: s_mov_b32 flat_scratch_lo, s13
; GCN-HSA-NEXT: s_waitcnt lgkmcnt(0)
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
-; GCN-HSA-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
; GCN-HSA-NEXT: s_add_u32 s4, s2, 0x70
+; GCN-HSA-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s5
+; GCN-HSA-NEXT: flat_load_dwordx4 v[24:27], v[0:1]
; GCN-HSA-NEXT: s_add_u32 s4, s2, 0x60
-; GCN-HSA-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s5
+; GCN-HSA-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
; GCN-HSA-NEXT: s_add_u32 s4, s2, 0x50
; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
-; GCN-HSA-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
; GCN-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCN-HSA-NEXT: v_mov_b32_e32 v1, s5
; GCN-HSA-NEXT: s_add_u32 s4, s2, 64
+; GCN-HSA-NEXT: flat_load_dwordx4 v[0:3], v[0:1]
; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
-; GCN-HSA-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s4
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s5
+; GCN-HSA-NEXT: v_mov_b32_e32 v9, s5
+; GCN-HSA-NEXT: v_mov_b32_e32 v8, s4
; GCN-HSA-NEXT: s_add_u32 s4, s2, 48
+; GCN-HSA-NEXT: flat_load_dwordx4 v[20:23], v[8:9]
; GCN-HSA-NEXT: s_addc_u32 s5, s3, 0
; GCN-HSA-NEXT: s_add_u32 s6, s2, 32
; GCN-HSA-NEXT: s_addc_u32 s7, s3, 0
-; GCN-HSA-NEXT: flat_load_dwordx4 v[0:3], v[0:1]
; GCN-HSA-NEXT: s_add_u32 s2, s2, 16
-; GCN-HSA-NEXT: s_addc_u32 s3, s3, 0
-; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
-; GCN-HSA-NEXT: flat_load_dwordx4 v[24:27], v[8:9]
; GCN-HSA-NEXT: v_mov_b32_e32 v9, s5
; GCN-HSA-NEXT: v_mov_b32_e32 v8, s4
+; GCN-HSA-NEXT: s_addc_u32 s3, s3, 0
+; GCN-HSA-NEXT: flat_load_dwordx4 v[12:15], v[8:9]
+; GCN-HSA-NEXT: v_mov_b32_e32 v9, s7
+; GCN-HSA-NEXT: v_mov_b32_e32 v8, s6
+; GCN-HSA-NEXT: v_mov_b32_e32 v17, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v16, s2
; GCN-HSA-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GCN-HSA-NEXT: v_mov_b32_e32 v17, s7
-; GCN-HSA-NEXT: v_mov_b32_e32 v16, s6
; GCN-HSA-NEXT: flat_load_dwordx4 v[16:19], v[16:17]
; GCN-HSA-NEXT: s_add_u32 s2, s0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
@@ -4386,143 +4344,144 @@ define amdgpu_kernel void @global_sextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-HSA-NEXT: v_mov_b32_e32 v29, s3
; GCN-HSA-NEXT: v_mov_b32_e32 v28, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xe0
-; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCN-HSA-NEXT: flat_store_dwordx4 v[36:37], v[32:35]
-; GCN-HSA-NEXT: v_mov_b32_e32 v36, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v35, s2
-; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xf0
+; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCN-HSA-NEXT: v_ashrrev_i32_e32 v34, 16, v31
; GCN-HSA-NEXT: v_ashrrev_i32_e32 v32, 16, v30
; GCN-HSA-NEXT: v_bfe_i32 v33, v31, 0, 16
; GCN-HSA-NEXT: v_bfe_i32 v31, v30, 0, 16
-; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCN-HSA-NEXT: flat_store_dwordx4 v[28:29], v[31:34]
; GCN-HSA-NEXT: s_waitcnt vmcnt(8)
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v29, 16, v20
-; GCN-HSA-NEXT: v_mov_b32_e32 v33, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v32, s2
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v29, 16, v24
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v31, 16, v25
+; GCN-HSA-NEXT: v_bfe_i32 v30, v25, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v28, v24, 0, 16
+; GCN-HSA-NEXT: v_mov_b32_e32 v25, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v24, s2
+; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xf0
+; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCN-HSA-NEXT: flat_store_dwordx4 v[24:25], v[28:31]
+; GCN-HSA-NEXT: v_mov_b32_e32 v25, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v24, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xc0
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v31, 16, v21
-; GCN-HSA-NEXT: v_bfe_i32 v30, v21, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v28, v20, 0, 16
-; GCN-HSA-NEXT: flat_store_dwordx4 v[35:36], v[28:31]
-; GCN-HSA-NEXT: v_mov_b32_e32 v35, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v34, s2
+; GCN-HSA-NEXT: v_mov_b32_e32 v32, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v31, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xd0
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v30, 16, v27
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v28, 16, v26
+; GCN-HSA-NEXT: v_bfe_i32 v29, v27, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v27, v26, 0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_mov_b32_e32 v37, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v36, s2
+; GCN-HSA-NEXT: flat_store_dwordx4 v[24:25], v[27:30]
+; GCN-HSA-NEXT: s_waitcnt vmcnt(9)
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v25, 16, v4
+; GCN-HSA-NEXT: v_mov_b32_e32 v29, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v28, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xa0
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v31, 16, v23
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v29, 16, v22
-; GCN-HSA-NEXT: v_bfe_i32 v30, v23, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v28, v22, 0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[32:33], v[28:31]
-; GCN-HSA-NEXT: v_mov_b32_e32 v33, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v32, s2
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v27, 16, v5
+; GCN-HSA-NEXT: v_bfe_i32 v26, v5, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v24, v4, 0, 16
+; GCN-HSA-NEXT: flat_store_dwordx4 v[31:32], v[24:27]
+; GCN-HSA-NEXT: v_mov_b32_e32 v31, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v30, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0xb0
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_mov_b32_e32 v39, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v38, s2
+; GCN-HSA-NEXT: v_mov_b32_e32 v33, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v32, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x80
-; GCN-HSA-NEXT: s_waitcnt vmcnt(9)
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v23, 16, v13
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v21, 16, v12
-; GCN-HSA-NEXT: v_bfe_i32 v22, v13, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v20, v12, 0, 16
-; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v31, 16, v15
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v29, 16, v14
-; GCN-HSA-NEXT: v_bfe_i32 v30, v15, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v28, v14, 0, 16
-; GCN-HSA-NEXT: flat_store_dwordx4 v[34:35], v[20:23]
-; GCN-HSA-NEXT: flat_store_dwordx4 v[36:37], v[28:31]
-; GCN-HSA-NEXT: s_waitcnt vmcnt(10)
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v15, 16, v5
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v13, 16, v4
-; GCN-HSA-NEXT: v_bfe_i32 v14, v5, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v12, v4, 0, 16
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v23, 16, v7
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v21, 16, v6
-; GCN-HSA-NEXT: v_bfe_i32 v22, v7, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v20, v6, 0, 16
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v27, 16, v7
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v25, 16, v6
+; GCN-HSA-NEXT: v_bfe_i32 v26, v7, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v24, v6, 0, 16
; GCN-HSA-NEXT: s_waitcnt vmcnt(9)
; GCN-HSA-NEXT: v_ashrrev_i32_e32 v7, 16, v1
; GCN-HSA-NEXT: v_ashrrev_i32_e32 v5, 16, v0
; GCN-HSA-NEXT: v_bfe_i32 v6, v1, 0, 16
; GCN-HSA-NEXT: v_bfe_i32 v4, v0, 0, 16
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
+; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCN-HSA-NEXT: flat_store_dwordx4 v[28:29], v[24:27]
+; GCN-HSA-NEXT: s_waitcnt vmcnt(9)
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v1, 16, v20
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v27, 16, v3
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v25, 16, v2
+; GCN-HSA-NEXT: v_bfe_i32 v26, v3, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v24, v2, 0, 16
+; GCN-HSA-NEXT: flat_store_dwordx4 v[30:31], v[4:7]
+; GCN-HSA-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
+; GCN-HSA-NEXT: v_mov_b32_e32 v5, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v4, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x90
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v3, 16, v21
+; GCN-HSA-NEXT: v_bfe_i32 v2, v21, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v0, v20, 0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
+; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GCN-HSA-NEXT: v_mov_b32_e32 v5, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v4, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x60
-; GCN-HSA-NEXT: flat_store_dwordx4 v[32:33], v[12:15]
-; GCN-HSA-NEXT: flat_store_dwordx4 v[38:39], v[20:23]
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v3, 16, v23
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v1, 16, v22
+; GCN-HSA-NEXT: v_bfe_i32 v2, v23, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v0, v22, 0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v23, 16, v3
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v21, 16, v2
-; GCN-HSA-NEXT: v_bfe_i32 v22, v3, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v20, v2, 0, 16
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[20:23]
-; GCN-HSA-NEXT: s_waitcnt vmcnt(11)
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v3, 16, v9
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v1, 16, v8
-; GCN-HSA-NEXT: v_bfe_i32 v2, v9, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v0, v8, 0, 16
-; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
+; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GCN-HSA-NEXT: v_mov_b32_e32 v5, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v4, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x70
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[0:3]
-; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
-; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
+; GCN-HSA-NEXT: s_waitcnt vmcnt(12)
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v3, 16, v13
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v1, 16, v12
+; GCN-HSA-NEXT: v_bfe_i32 v2, v13, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v0, v12, 0, 16
+; GCN-HSA-NEXT: v_mov_b32_e32 v13, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v12, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 64
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v13, 16, v24
-; GCN-HSA-NEXT: v_bfe_i32 v12, v24, 0, 16
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v24, 16, v11
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v22, 16, v10
-; GCN-HSA-NEXT: v_bfe_i32 v23, v11, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v21, v10, 0, 16
+; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
+; GCN-HSA-NEXT: s_waitcnt vmcnt(11)
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v7, 16, v17
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v1, 16, v18
+; GCN-HSA-NEXT: v_bfe_i32 v0, v18, 0, 16
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v5, 16, v16
+; GCN-HSA-NEXT: v_bfe_i32 v6, v17, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v4, v16, 0, 16
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v18, 16, v15
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v16, 16, v14
+; GCN-HSA-NEXT: v_bfe_i32 v17, v15, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v15, v14, 0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[21:24]
+; GCN-HSA-NEXT: flat_store_dwordx4 v[12:13], v[15:18]
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v14, 16, v11
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v18, 16, v9
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v16, 16, v8
+; GCN-HSA-NEXT: v_bfe_i32 v17, v9, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v15, v8, 0, 16
; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 0x50
-; GCN-HSA-NEXT: s_waitcnt vmcnt(12)
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v3, 16, v19
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v1, 16, v18
-; GCN-HSA-NEXT: v_bfe_i32 v2, v19, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v0, v18, 0, 16
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v20, 16, v17
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v18, 16, v16
-; GCN-HSA-NEXT: v_bfe_i32 v19, v17, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v17, v16, 0, 16
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[17:20]
+; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[15:18]
; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
; GCN-HSA-NEXT: s_add_u32 s2, s0, 32
; GCN-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v12, 16, v10
+; GCN-HSA-NEXT: v_bfe_i32 v13, v11, 0, 16
+; GCN-HSA-NEXT: v_bfe_i32 v11, v10, 0, 16
; GCN-HSA-NEXT: s_add_u32 s0, s0, 48
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v15, 16, v25
-; GCN-HSA-NEXT: v_bfe_i32 v14, v25, 0, 16
-; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[0:3]
+; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[11:14]
+; GCN-HSA-NEXT: v_mov_b32_e32 v9, s3
+; GCN-HSA-NEXT: v_mov_b32_e32 v8, s2
; GCN-HSA-NEXT: s_addc_u32 s1, s1, 0
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s2
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s3
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v7, 16, v27
-; GCN-HSA-NEXT: v_ashrrev_i32_e32 v5, 16, v26
-; GCN-HSA-NEXT: v_bfe_i32 v6, v27, 0, 16
-; GCN-HSA-NEXT: v_bfe_i32 v4, v26, 0, 16
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[12:15]
-; GCN-HSA-NEXT: v_mov_b32_e32 v0, s0
-; GCN-HSA-NEXT: v_mov_b32_e32 v1, s1
-; GCN-HSA-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
+; GCN-HSA-NEXT: v_ashrrev_i32_e32 v3, 16, v19
+; GCN-HSA-NEXT: v_bfe_i32 v2, v19, 0, 16
+; GCN-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
+; GCN-HSA-NEXT: s_nop 0
+; GCN-HSA-NEXT: v_mov_b32_e32 v5, s1
+; GCN-HSA-NEXT: v_mov_b32_e32 v4, s0
+; GCN-HSA-NEXT: flat_store_dwordx4 v[4:5], v[0:3]
; GCN-HSA-NEXT: s_endpgm
;
; GCN-NOHSA-VI-LABEL: global_sextload_v64i16_to_v64i32:
@@ -4540,113 +4499,102 @@ define amdgpu_kernel void @global_sextload_v64i16_to_v64i32(ptr addrspace(1) %ou
; GCN-NOHSA-VI-NEXT: s_mov_b32 s8, s6
; GCN-NOHSA-VI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-VI-NEXT: s_mov_b32 s11, s3
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:16
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:32
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:48
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[23:26], off, s[8:11], 0 offset:64
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[27:30], off, s[8:11], 0 offset:80
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[31:34], off, s[8:11], 0 offset:96
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[35:38], off, s[8:11], 0 offset:112
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0 offset:16
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[16:19], off, s[8:11], 0 offset:32
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[20:23], off, s[8:11], 0 offset:48
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[24:27], off, s[8:11], 0 offset:64
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:80
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[32:35], off, s[8:11], 0 offset:96
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[36:39], off, s[8:11], 0 offset:112
; GCN-NOHSA-VI-NEXT: s_addc_u32 s89, s89, 0
; GCN-NOHSA-VI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-VI-NEXT: s_mov_b32 s1, s5
; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(7)
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v18, 16, v15
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v16, 16, v14
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v17, v15, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v15, v14, 0, 16
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v15, off, s[88:91], 0 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v16, off, s[88:91], 0 offset:4 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v17, off, s[88:91], 0 offset:8 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v18, off, s[88:91], 0 offset:12 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v16, 16, v13
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v14, 16, v12
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v15, v13, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v13, v12, 0, 16
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v13, off, s[88:91], 0 offset:16 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v14, off, s[88:91], 0 offset:20 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v15, off, s[88:91], 0 offset:24 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: buffer_store_dword v16, off, s[88:91], 0 offset:28 ; 4-byte Folded Spill
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(14)
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v19, 16, v11
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v17, 16, v10
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v18, v11, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v16, v10, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v42, 16, v9
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v40, 16, v8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v41, v9, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v39, v8, 0, 16
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(12)
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v11, 16, v3
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v9, 16, v2
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v10, v3, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v8, v2, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v50, 16, v1
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v48, 16, v0
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v49, v1, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v47, v0, 0, 16
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(8)
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v3, 16, v36
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v1, 16, v35
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v2, v36, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v0, v35, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v15, 16, v7
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v13, 16, v6
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v14, v7, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v12, v6, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v46, 16, v5
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v44, 16, v4
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v45, v5, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v43, v4, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v54, 16, v26
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v52, 16, v25
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v53, v26, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v51, v25, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v58, 16, v24
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v56, 16, v23
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v57, v24, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v55, v23, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v26, 16, v30
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v24, 16, v29
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v25, v30, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v23, v29, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v62, 16, v28
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v60, 16, v27
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v61, v28, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v59, v27, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v30, 16, v34
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v28, 16, v33
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v29, v34, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v27, v33, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v7, 16, v32
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v5, 16, v31
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v6, v32, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v4, v31, 0, 16
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v34, 16, v38
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v32, 16, v37
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v33, v38, 0, 16
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v31, v37, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v3, 16, v11
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v1, 16, v10
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v2, v11, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v0, v10, 0, 16
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v0, off, s[88:91], 0 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v1, off, s[88:91], 0 offset:4 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v2, off, s[88:91], 0 offset:8 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: buffer_store_dword v3, off, s[88:91], 0 offset:12 ; 4-byte Folded Spill
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v7, 16, v9
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v5, 16, v8
+; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(4)
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v3, 16, v37
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v1, 16, v36
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v2, v37, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v0, v36, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v6, v9, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v4, v8, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v11, 16, v15
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v9, 16, v14
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v10, v15, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v8, v14, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v43, 16, v13
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v41, 16, v12
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v42, v13, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v40, v12, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v15, 16, v19
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v13, 16, v18
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v14, v19, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v12, v18, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v47, 16, v17
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v45, 16, v16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v46, v17, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v44, v16, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v19, 16, v23
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v17, 16, v22
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v18, v23, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v16, v22, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v51, 16, v21
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v49, 16, v20
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v50, v21, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v48, v20, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v23, 16, v27
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v21, 16, v26
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v22, v27, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v20, v26, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v55, 16, v25
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v53, 16, v24
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v54, v25, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v52, v24, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v27, 16, v31
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v25, 16, v30
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v26, v31, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v24, v30, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v59, 16, v29
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v57, 16, v28
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v58, v29, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v56, v28, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v31, 16, v35
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v29, 16, v34
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v30, v35, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v28, v34, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v63, 16, v33
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v61, 16, v32
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v62, v33, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v60, v32, 0, 16
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v35, 16, v39
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v33, 16, v38
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v34, v39, 0, 16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v32, v38, 0, 16
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[31:34], off, s[0:3], 0 offset:240
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:192
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[27:30], off, s[0:3], 0 offset:208
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[59:62], off, s[0:3], 0 offset:160
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[23:26], off, s[0:3], 0 offset:176
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[55:58], off, s[0:3], 0 offset:128
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[51:54], off, s[0:3], 0 offset:144
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[47:50], off, s[0:3], 0 offset:96
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:112
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[43:46], off, s[0:3], 0 offset:64
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:240
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[60:63], off, s[0:3], 0 offset:192
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:208
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[56:59], off, s[0:3], 0 offset:160
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:176
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[52:55], off, s[0:3], 0 offset:128
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:144
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[48:51], off, s[0:3], 0 offset:96
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:112
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[44:47], off, s[0:3], 0 offset:64
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:80
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[39:42], off, s[0:3], 0 offset:32
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:48
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v0, off, s[88:91], 0 offset:16 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v1, off, s[88:91], 0 offset:20 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v2, off, s[88:91], 0 offset:24 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: buffer_load_dword v3, off, s[88:91], 0 offset:28 ; 4-byte Folded Reload
-; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(0)
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[40:43], off, s[0:3], 0 offset:32
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0
; GCN-NOHSA-VI-NEXT: buffer_load_dword v0, off, s[88:91], 0 ; 4-byte Folded Reload
; GCN-NOHSA-VI-NEXT: buffer_load_dword v1, off, s[88:91], 0 offset:4 ; 4-byte Folded Reload
; GCN-NOHSA-VI-NEXT: buffer_load_dword v2, off, s[88:91], 0 offset:8 ; 4-byte Folded Reload
diff --git a/llvm/test/CodeGen/AMDGPU/load-global-i32.ll b/llvm/test/CodeGen/AMDGPU/load-global-i32.ll
index 930fe409c2cea..38d5584907c22 100644
--- a/llvm/test/CodeGen/AMDGPU/load-global-i32.ll
+++ b/llvm/test/CodeGen/AMDGPU/load-global-i32.ll
@@ -2342,56 +2342,56 @@ define amdgpu_kernel void @global_sextload_v16i32_to_v16i64(ptr addrspace(1) %ou
; SI-NOHSA-NEXT: s_waitcnt lgkmcnt(0)
; SI-NOHSA-NEXT: s_mov_b32 s8, s6
; SI-NOHSA-NEXT: s_mov_b32 s9, s7
-; SI-NOHSA-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:32
-; SI-NOHSA-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:48
-; SI-NOHSA-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:16
-; SI-NOHSA-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0
+; SI-NOHSA-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:16
+; SI-NOHSA-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:32
+; SI-NOHSA-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:48
; SI-NOHSA-NEXT: s_mov_b32 s0, s4
; SI-NOHSA-NEXT: s_mov_b32 s1, s5
-; SI-NOHSA-NEXT: s_waitcnt vmcnt(3)
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v19, 31, v3
; SI-NOHSA-NEXT: s_waitcnt vmcnt(2)
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v27, 31, v5
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v25, 31, v4
-; SI-NOHSA-NEXT: v_mov_b32_e32 v24, v4
-; SI-NOHSA-NEXT: v_mov_b32_e32 v26, v5
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v17, 31, v2
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v23, 31, v7
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v21, 31, v6
-; SI-NOHSA-NEXT: v_mov_b32_e32 v20, v6
-; SI-NOHSA-NEXT: v_mov_b32_e32 v22, v7
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v14, 31, v3
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v12, 31, v2
+; SI-NOHSA-NEXT: s_waitcnt vmcnt(1)
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v18, 31, v7
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v16, 31, v6
+; SI-NOHSA-NEXT: s_waitcnt vmcnt(0)
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v22, 31, v11
+; SI-NOHSA-NEXT: v_mov_b32_e32 v21, v11
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v30, 31, v5
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v28, 31, v4
+; SI-NOHSA-NEXT: v_mov_b32_e32 v27, v4
+; SI-NOHSA-NEXT: v_mov_b32_e32 v29, v5
+; SI-NOHSA-NEXT: v_mov_b32_e32 v15, v6
+; SI-NOHSA-NEXT: v_mov_b32_e32 v17, v7
; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v7, 31, v1
; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v5, 31, v0
; SI-NOHSA-NEXT: v_mov_b32_e32 v4, v0
; SI-NOHSA-NEXT: v_mov_b32_e32 v6, v1
-; SI-NOHSA-NEXT: v_mov_b32_e32 v16, v2
-; SI-NOHSA-NEXT: v_mov_b32_e32 v18, v3
-; SI-NOHSA-NEXT: s_waitcnt vmcnt(1)
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v3, 31, v11
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v1, 31, v10
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v31, 31, v9
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v29, 31, v8
-; SI-NOHSA-NEXT: v_mov_b32_e32 v28, v8
-; SI-NOHSA-NEXT: v_mov_b32_e32 v30, v9
-; SI-NOHSA-NEXT: v_mov_b32_e32 v0, v10
-; SI-NOHSA-NEXT: v_mov_b32_e32 v2, v11
+; SI-NOHSA-NEXT: v_mov_b32_e32 v11, v2
+; SI-NOHSA-NEXT: v_mov_b32_e32 v13, v3
+; SI-NOHSA-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v26, 31, v9
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v24, 31, v8
+; SI-NOHSA-NEXT: v_mov_b32_e32 v23, v8
+; SI-NOHSA-NEXT: v_mov_b32_e32 v25, v9
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v20, 31, v10
+; SI-NOHSA-NEXT: v_mov_b32_e32 v19, v10
; SI-NOHSA-NEXT: s_waitcnt vmcnt(0)
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v11, 31, v15
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v9, 31, v14
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v35, 31, v13
-; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v33, 31, v12
-; SI-NOHSA-NEXT: v_mov_b32_e32 v32, v12
-; SI-NOHSA-NEXT: v_mov_b32_e32 v34, v13
-; SI-NOHSA-NEXT: v_mov_b32_e32 v8, v14
-; SI-NOHSA-NEXT: v_mov_b32_e32 v10, v15
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:96
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:112
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:64
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:80
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:32
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:48
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0
-; SI-NOHSA-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:16
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v34, 31, v3
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v32, 31, v2
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v38, 31, v1
+; SI-NOHSA-NEXT: v_ashrrev_i32_e32 v36, 31, v0
+; SI-NOHSA-NEXT: v_mov_b32_e32 v35, v0
+; SI-NOHSA-NEXT: v_mov_b32_e32 v37, v1
+; SI-NOHSA-NEXT: v_mov_b32_e32 v31, v2
+; SI-NOHSA-NEXT: v_mov_b32_e32 v33, v3
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[23:26], off, s[0:3], 0 offset:96
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[19:22], off, s[0:3], 0 offset:112
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[27:30], off, s[0:3], 0 offset:64
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[15:18], off, s[0:3], 0 offset:80
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:32
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[11:14], off, s[0:3], 0 offset:48
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[35:38], off, s[0:3], 0
+; SI-NOHSA-NEXT: buffer_store_dwordx4 v[31:34], off, s[0:3], 0 offset:16
; SI-NOHSA-NEXT: s_endpgm
;
; GCNX3-HSA-LABEL: global_sextload_v16i32_to_v16i64:
@@ -2505,56 +2505,56 @@ define amdgpu_kernel void @global_sextload_v16i32_to_v16i64(ptr addrspace(1) %ou
; GCNX3-NOHSA-NEXT: s_waitcnt lgkmcnt(0)
; GCNX3-NOHSA-NEXT: s_mov_b32 s8, s6
; GCNX3-NOHSA-NEXT: s_mov_b32 s9, s7
-; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:32
-; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:48
-; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:16
-; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[12:15], off, s[8:11], 0
+; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0 offset:16
+; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[4:7], off, s[8:11], 0 offset:32
+; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[8:11], off, s[8:11], 0 offset:48
; GCNX3-NOHSA-NEXT: s_mov_b32 s0, s4
; GCNX3-NOHSA-NEXT: s_mov_b32 s1, s5
-; GCNX3-NOHSA-NEXT: s_waitcnt vmcnt(3)
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v19, 31, v3
; GCNX3-NOHSA-NEXT: s_waitcnt vmcnt(2)
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v27, 31, v5
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v25, 31, v4
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v24, v4
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v26, v5
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v17, 31, v2
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v23, 31, v7
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v21, 31, v6
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v20, v6
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v22, v7
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v14, 31, v3
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v12, 31, v2
+; GCNX3-NOHSA-NEXT: s_waitcnt vmcnt(1)
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v18, 31, v7
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v16, 31, v6
+; GCNX3-NOHSA-NEXT: s_waitcnt vmcnt(0)
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v22, 31, v11
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v21, v11
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v30, 31, v5
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v28, 31, v4
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v27, v4
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v29, v5
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v15, v6
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v17, v7
; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v7, 31, v1
; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v5, 31, v0
; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v4, v0
; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v6, v1
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v16, v2
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v18, v3
-; GCNX3-NOHSA-NEXT: s_waitcnt vmcnt(1)
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v3, 31, v11
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v1, 31, v10
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v31, 31, v9
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v29, 31, v8
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v28, v8
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v30, v9
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v0, v10
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v2, v11
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v11, v2
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v13, v3
+; GCNX3-NOHSA-NEXT: buffer_load_dwordx4 v[0:3], off, s[8:11], 0
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v26, 31, v9
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v24, 31, v8
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v23, v8
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v25, v9
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v20, 31, v10
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v19, v10
; GCNX3-NOHSA-NEXT: s_waitcnt vmcnt(0)
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v11, 31, v15
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v9, 31, v14
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v35, 31, v13
-; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v33, 31, v12
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v32, v12
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v34, v13
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v8, v14
-; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v10, v15
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:96
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:112
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:64
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:80
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:32
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:48
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0
-; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:16
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v34, 31, v3
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v32, 31, v2
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v38, 31, v1
+; GCNX3-NOHSA-NEXT: v_ashrrev_i32_e32 v36, 31, v0
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v35, v0
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v37, v1
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v31, v2
+; GCNX3-NOHSA-NEXT: v_mov_b32_e32 v33, v3
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[23:26], off, s[0:3], 0 offset:96
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[19:22], off, s[0:3], 0 offset:112
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[27:30], off, s[0:3], 0 offset:64
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[15:18], off, s[0:3], 0 offset:80
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0 offset:32
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[11:14], off, s[0:3], 0 offset:48
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[35:38], off, s[0:3], 0
+; GCNX3-NOHSA-NEXT: buffer_store_dwordx4 v[31:34], off, s[0:3], 0 offset:16
; GCNX3-NOHSA-NEXT: s_endpgm
;
; EG-LABEL: global_sextload_v16i32_to_v16i64:
@@ -3121,13 +3121,13 @@ define amdgpu_kernel void @global_sextload_v32i32_to_v32i64(ptr addrspace(1) %ou
; GCNX3-HSA: ; %bb.0:
; GCNX3-HSA-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0
; GCNX3-HSA-NEXT: s_add_i32 s12, s12, s17
-; GCNX3-HSA-NEXT: s_mov_b32 flat_scratch_lo, s13
; GCNX3-HSA-NEXT: s_lshr_b32 flat_scratch_hi, s12, 8
+; GCNX3-HSA-NEXT: s_mov_b32 flat_scratch_lo, s13
; GCNX3-HSA-NEXT: s_waitcnt lgkmcnt(0)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s2
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s3
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
; GCNX3-HSA-NEXT: s_add_u32 s4, s2, 0x70
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
; GCNX3-HSA-NEXT: s_addc_u32 s5, s3, 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s5
@@ -3137,31 +3137,31 @@ define amdgpu_kernel void @global_sextload_v32i32_to_v32i64(ptr addrspace(1) %ou
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s5
; GCNX3-HSA-NEXT: s_add_u32 s4, s2, 0x50
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[8:11], v[0:1]
; GCNX3-HSA-NEXT: s_addc_u32 s5, s3, 0
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s5
; GCNX3-HSA-NEXT: s_add_u32 s4, s2, 64
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
; GCNX3-HSA-NEXT: s_addc_u32 s5, s3, 0
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s4
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s5
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[4:7], v[0:1]
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GCNX3-HSA-NEXT: s_add_u32 s4, s2, 48
; GCNX3-HSA-NEXT: s_addc_u32 s5, s3, 0
; GCNX3-HSA-NEXT: s_add_u32 s6, s2, 32
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s5
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s4
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s4
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s5
; GCNX3-HSA-NEXT: s_addc_u32 s7, s3, 0
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[16:19], v[8:9]
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[16:19], v[0:1]
; GCNX3-HSA-NEXT: s_add_u32 s2, s2, 16
; GCNX3-HSA-NEXT: s_addc_u32 s3, s3, 0
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s2
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s7
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s6
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[0:3], v[0:1]
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s6
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s7
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, s2
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v3, s3
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[0:3], v[2:3]
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 16
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v37, s1
@@ -3174,143 +3174,143 @@ define amdgpu_kernel void @global_sextload_v32i32_to_v32i64(ptr addrspace(1) %ou
; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xe0
-; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[36:37], v[32:35]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v37, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v36, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xf0
+; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v35, 31, v31
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v33, 31, v30
; GCNX3-HSA-NEXT: v_mov_b32_e32 v32, v30
; GCNX3-HSA-NEXT: v_mov_b32_e32 v34, v31
-; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[32:35]
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(8)
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v31, 31, v25
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v33, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v32, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xc0
-; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v35, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v34, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xd0
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v29, 31, v24
; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, v24
; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, v25
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v25, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xf0
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[36:37], v[28:31]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v37, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v36, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xa0
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[24:25], v[28:31]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v25, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xc0
+; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v33, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v32, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xd0
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v31, 31, v27
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v29, 31, v26
; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, v26
; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, v27
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[32:33], v[28:31]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v33, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v32, s2
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[24:25], v[28:31]
+; GCNX3-HSA-NEXT: s_waitcnt vmcnt(9)
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v27, 31, v9
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xa0
+; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v31, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xb0
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v25, 31, v8
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, v8
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v26, v9
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v39, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v38, s2
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v33, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v32, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x80
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v27, 31, v11
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v25, 31, v10
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, v10
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v26, v11
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(9)
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v27, 31, v21
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v25, 31, v20
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, v20
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v26, v21
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v11, 31, v5
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v9, 31, v4
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, v4
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v10, v5
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v31, 31, v23
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v29, 31, v22
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, v22
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, v23
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[34:35], v[24:27]
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[36:37], v[28:31]
-; GCNX3-HSA-NEXT: s_waitcnt vmcnt(10)
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v23, 31, v15
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v21, 31, v14
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v27, 31, v13
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v25, 31, v12
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, v12
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v26, v13
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v20, v14
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v22, v15
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[24:27]
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(9)
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v15, 31, v5
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v13, 31, v4
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v12, v4
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v14, v5
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v5, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, s2
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v5, 31, v20
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v27, 31, v7
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v25, 31, v6
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, v6
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v26, v7
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[30:31], v[8:11]
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x90
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v7, 31, v21
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, v20
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v6, v21
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[4:5], v[12:15]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v5, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, s2
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x60
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[32:33], v[24:27]
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[38:39], v[20:23]
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v26, 31, v7
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v24, 31, v6
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v23, v6
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v25, v7
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v7, 31, v23
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v5, 31, v22
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, v22
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v6, v23
+; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x70
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[4:5], v[23:26]
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(12)
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v7, 31, v17
; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v5, 31, v16
; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, v16
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v16, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v15, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x70
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v7, 31, v17
; GCNX3-HSA-NEXT: v_mov_b32_e32 v6, v17
-; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[15:16], v[4:7]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v16, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v15, s2
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v17, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v16, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 64
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v26, 31, v19
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v24, 31, v18
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v23, v18
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v25, v19
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v11, 31, v19
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v9, 31, v18
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, v18
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v10, v19
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[15:16], v[23:26]
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[16:17], v[8:11]
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(12)
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v18, 31, v9
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v16, 31, v8
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v15, v8
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v17, v9
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s2
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v7, 31, v3
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v11, 31, v13
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v9, 31, v12
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, v12
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v10, v13
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v13, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v12, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x50
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[8:9], v[15:18]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s2
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[12:13], v[8:11]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v13, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v12, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 32
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: s_add_u32 s0, s0, 48
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v14, 31, v1
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v12, 31, v0
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v7, 31, v11
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v6, v11
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v11, v0
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v13, v1
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v11, 31, v1
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v9, 31, v0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, v0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v10, v1
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s2
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s3
; GCNX3-HSA-NEXT: s_addc_u32 s1, s1, 0
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v22, 31, v3
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v20, 31, v2
-; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v5, 31, v10
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, v10
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[0:1], v[11:14]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v19, v2
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v21, v3
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v5, 31, v2
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v19, 31, v15
+; GCNX3-HSA-NEXT: v_ashrrev_i32_e32 v17, 31, v14
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v16, v14
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v18, v15
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[0:1], v[8:11]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, v2
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v6, v3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s1
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[8:9], v[4:7]
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[0:1], v[19:22]
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[12:13], v[16:19]
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[0:1], v[4:7]
; GCNX3-HSA-NEXT: s_endpgm
;
; GCNX3-NOHSA-LABEL: global_sextload_v32i32_to_v32i64:
@@ -3901,13 +3901,13 @@ define amdgpu_kernel void @global_zextload_v32i32_to_v32i64(ptr addrspace(1) %ou
; GCNX3-HSA-NEXT: s_addc_u32 s9, s3, 0
; GCNX3-HSA-NEXT: s_add_u32 s10, s2, 64
; GCNX3-HSA-NEXT: s_addc_u32 s11, s3, 0
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s2
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s3
; GCNX3-HSA-NEXT: s_add_u32 s12, s2, 0x50
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
; GCNX3-HSA-NEXT: s_addc_u32 s13, s3, 0
; GCNX3-HSA-NEXT: s_add_u32 s14, s2, 0x60
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s2
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s3
; GCNX3-HSA-NEXT: s_addc_u32 s15, s3, 0
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[28:31], v[0:1]
; GCNX3-HSA-NEXT: s_add_u32 s2, s2, 0x70
; GCNX3-HSA-NEXT: s_addc_u32 s3, s3, 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s2
@@ -3918,113 +3918,114 @@ define amdgpu_kernel void @global_zextload_v32i32_to_v32i64(ptr addrspace(1) %ou
; GCNX3-HSA-NEXT: flat_load_dwordx4 v[24:27], v[0:1]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s12
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s13
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[16:19], v[0:1]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s10
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s11
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[16:19], v[0:1]
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[20:23], v[0:1]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, s8
; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, s9
; GCNX3-HSA-NEXT: flat_load_dwordx4 v[12:15], v[0:1]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v4, s4
; GCNX3-HSA-NEXT: v_mov_b32_e32 v5, s5
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s7
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v8, s6
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v6, s6
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v7, s7
+; GCNX3-HSA-NEXT: flat_load_dwordx4 v[8:11], v[6:7]
; GCNX3-HSA-NEXT: flat_load_dwordx4 v[4:7], v[4:5]
-; GCNX3-HSA-NEXT: flat_load_dwordx4 v[8:11], v[8:9]
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, 0
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 16
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v3, v1
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v1, 0
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v3, v1
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v37, s1
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v36, s0
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(7)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v28
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v29
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s1
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s0
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xe0
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[36:37], v[0:3]
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v30
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v31
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v31, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, s2
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xf0
+; GCNX3-HSA-NEXT: s_waitcnt vmcnt(8)
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v32
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v33
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xc0
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: s_waitcnt vmcnt(8)
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v32
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v33
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[30:31], v[0:3]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v31, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xd0
-; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v33, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v32, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xa0
-; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v34
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v35
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v35, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v34, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xb0
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x80
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xa0
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(9)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v24
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v25
; GCNX3-HSA-NEXT: v_mov_b32_e32 v25, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v24, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x90
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[30:31], v[0:3]
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0xb0
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[30:31], v[0:3]
+; GCNX3-HSA-NEXT: s_nop 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v26
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v27
; GCNX3-HSA-NEXT: v_mov_b32_e32 v27, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v26, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x60
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[32:33], v[0:3]
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x80
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: s_waitcnt vmcnt(10)
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v20
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v21
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[34:35], v[0:3]
; GCNX3-HSA-NEXT: v_mov_b32_e32 v31, s3
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v22
-; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v23
; GCNX3-HSA-NEXT: v_mov_b32_e32 v30, s2
-; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x70
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x90
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
-; GCNX3-HSA-NEXT: s_waitcnt vmcnt(11)
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v29, s3
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v28, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x60
+; GCNX3-HSA-NEXT: s_waitcnt vmcnt(10)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v16
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v17
+; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[24:25], v[0:3]
-; GCNX3-HSA-NEXT: s_nop 0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v17, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v18
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v19
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v16, s2
+; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 0x70
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[26:27], v[0:3]
+; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
+; GCNX3-HSA-NEXT: s_waitcnt vmcnt(11)
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v20
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v21
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[30:31], v[0:3]
+; GCNX3-HSA-NEXT: s_nop 0
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v22
+; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v23
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[28:29], v[0:3]
; GCNX3-HSA-NEXT: s_waitcnt vmcnt(12)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v12
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v13
; GCNX3-HSA-NEXT: v_mov_b32_e32 v13, s3
; GCNX3-HSA-NEXT: v_mov_b32_e32 v12, s2
; GCNX3-HSA-NEXT: s_add_u32 s2, s0, 64
-; GCNX3-HSA-NEXT: flat_store_dwordx4 v[30:31], v[0:3]
+; GCNX3-HSA-NEXT: flat_store_dwordx4 v[16:17], v[0:3]
; GCNX3-HSA-NEXT: s_addc_u32 s3, s1, 0
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v14
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v15
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[12:13], v[0:3]
-; GCNX3-HSA-NEXT: s_waitcnt vmcnt(12)
+; GCNX3-HSA-NEXT: s_waitcnt vmcnt(13)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v8
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v9
; GCNX3-HSA-NEXT: v_mov_b32_e32 v9, s3
@@ -4041,6 +4042,7 @@ define amdgpu_kernel void @global_zextload_v32i32_to_v32i64(ptr addrspace(1) %ou
; GCNX3-HSA-NEXT: s_add_u32 s0, s0, 48
; GCNX3-HSA-NEXT: flat_store_dwordx4 v[8:9], v[0:3]
; GCNX3-HSA-NEXT: s_addc_u32 s1, s1, 0
+; GCNX3-HSA-NEXT: s_waitcnt vmcnt(14)
; GCNX3-HSA-NEXT: v_mov_b32_e32 v0, v4
; GCNX3-HSA-NEXT: v_mov_b32_e32 v2, v5
; GCNX3-HSA-NEXT: v_mov_b32_e32 v5, s3
diff --git a/llvm/test/CodeGen/AMDGPU/load-global-i8.ll b/llvm/test/CodeGen/AMDGPU/load-global-i8.ll
index 8e179d1bfeb1b..9694484c2ed2e 100644
--- a/llvm/test/CodeGen/AMDGPU/load-global-i8.ll
+++ b/llvm/test/CodeGen/AMDGPU/load-global-i8.ll
@@ -3326,9 +3326,9 @@ define amdgpu_kernel void @global_zextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-SI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-SI-NEXT: s_mov_b32 s11, s3
; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[16:19], off, s[8:11], 0
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:16
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[32:35], off, s[8:11], 0 offset:32
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[36:39], off, s[8:11], 0 offset:48
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[20:23], off, s[8:11], 0 offset:16
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[24:27], off, s[8:11], 0 offset:32
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:48
; GCN-NOHSA-SI-NEXT: s_addc_u32 s13, s13, 0
; GCN-NOHSA-SI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-SI-NEXT: s_mov_b32 s1, s5
@@ -3342,10 +3342,10 @@ define amdgpu_kernel void @global_zextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-SI-NEXT: buffer_store_dword v2, off, s[12:15], 0 offset:8 ; 4-byte Folded Spill
; GCN-NOHSA-SI-NEXT: buffer_store_dword v3, off, s[12:15], 0 offset:12 ; 4-byte Folded Spill
; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(4) expcnt(0)
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v3, 24, v38
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v1, v38, 8, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v0, 0xff, v38
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v2, v38, 16, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v3, 24, v30
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v1, v30, 8, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v0, 0xff, v30
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v2, v30, 16, 8
; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v7, 24, v16
; GCN-NOHSA-SI-NEXT: v_bfe_u32 v5, v16, 8, 8
; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v11, 24, v19
@@ -3358,61 +3358,61 @@ define amdgpu_kernel void @global_zextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-SI-NEXT: v_bfe_u32 v10, v19, 16, 8
; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v12, 0xff, v18
; GCN-NOHSA-SI-NEXT: v_bfe_u32 v14, v18, 16, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v19, 24, v29
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v17, v29, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v23, 24, v28
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v21, v28, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v27, 24, v31
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v25, v31, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v43, 24, v30
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v41, v30, 8, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v16, 0xff, v29
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v18, v29, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v20, 0xff, v28
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v22, v28, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v24, 0xff, v31
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v26, v31, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v40, 0xff, v30
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v42, v30, 16, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v31, 24, v33
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v29, v33, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v47, 24, v32
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v45, v32, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v51, 24, v35
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v49, v35, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v55, 24, v34
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v53, v34, 8, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v28, 0xff, v33
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v30, v33, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v44, 0xff, v32
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v46, v32, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v48, 0xff, v35
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v50, v35, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v52, 0xff, v34
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v54, v34, 16, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v35, 24, v37
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v33, v37, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v59, 24, v36
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v57, v36, 8, 8
-; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v63, 24, v39
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v61, v39, 8, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v32, 0xff, v37
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v34, v37, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v56, 0xff, v36
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v58, v36, 16, 8
-; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v60, 0xff, v39
-; GCN-NOHSA-SI-NEXT: v_bfe_u32 v62, v39, 16, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v19, 24, v21
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v17, v21, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v35, 24, v20
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v33, v20, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v39, 24, v23
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v37, v23, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v43, 24, v22
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v41, v22, 8, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v16, 0xff, v21
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v18, v21, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v32, 0xff, v20
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v34, v20, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v36, 0xff, v23
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v38, v23, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v40, 0xff, v22
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v42, v22, 16, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v23, 24, v25
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v21, v25, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v47, 24, v24
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v45, v24, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v51, 24, v27
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v49, v27, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v55, 24, v26
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v53, v26, 8, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v20, 0xff, v25
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v22, v25, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v44, 0xff, v24
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v46, v24, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v48, 0xff, v27
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v50, v27, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v52, 0xff, v26
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v54, v26, 16, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v27, 24, v29
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v25, v29, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v59, 24, v28
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v57, v28, 8, 8
+; GCN-NOHSA-SI-NEXT: v_lshrrev_b32_e32 v63, 24, v31
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v61, v31, 8, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v24, 0xff, v29
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v26, v29, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v56, 0xff, v28
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v58, v28, 16, 8
+; GCN-NOHSA-SI-NEXT: v_and_b32_e32 v60, 0xff, v31
+; GCN-NOHSA-SI-NEXT: v_bfe_u32 v62, v31, 16, 8
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[60:63], off, s[0:3], 0 offset:240
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[56:59], off, s[0:3], 0 offset:192
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:208
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:208
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[52:55], off, s[0:3], 0 offset:160
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[48:51], off, s[0:3], 0 offset:176
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[44:47], off, s[0:3], 0 offset:128
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:144
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:144
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[40:43], off, s[0:3], 0 offset:96
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:112
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:64
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[36:39], off, s[0:3], 0 offset:112
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:64
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:80
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:32
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
@@ -3614,9 +3614,9 @@ define amdgpu_kernel void @global_zextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-VI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-VI-NEXT: s_mov_b32 s11, s3
; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[16:19], off, s[8:11], 0
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:16
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[32:35], off, s[8:11], 0 offset:32
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[36:39], off, s[8:11], 0 offset:48
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[20:23], off, s[8:11], 0 offset:16
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[24:27], off, s[8:11], 0 offset:32
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[28:31], off, s[8:11], 0 offset:48
; GCN-NOHSA-VI-NEXT: s_addc_u32 s89, s89, 0
; GCN-NOHSA-VI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-VI-NEXT: s_mov_b32 s1, s5
@@ -3630,10 +3630,10 @@ define amdgpu_kernel void @global_zextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-VI-NEXT: buffer_store_dword v2, off, s[88:91], 0 offset:8 ; 4-byte Folded Spill
; GCN-NOHSA-VI-NEXT: buffer_store_dword v3, off, s[88:91], 0 offset:12 ; 4-byte Folded Spill
; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(4)
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v3, 24, v38
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v1, v38, 8, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v0, 0xff, v38
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v2, v38, 16, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v3, 24, v30
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v1, v30, 8, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v0, 0xff, v30
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v2, v30, 16, 8
; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v7, 24, v16
; GCN-NOHSA-VI-NEXT: v_bfe_u32 v5, v16, 8, 8
; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v11, 24, v19
@@ -3646,61 +3646,61 @@ define amdgpu_kernel void @global_zextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-VI-NEXT: v_bfe_u32 v10, v19, 16, 8
; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v12, 0xff, v18
; GCN-NOHSA-VI-NEXT: v_bfe_u32 v14, v18, 16, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v19, 24, v29
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v17, v29, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v23, 24, v28
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v21, v28, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v27, 24, v31
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v25, v31, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v43, 24, v30
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v41, v30, 8, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v16, 0xff, v29
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v18, v29, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v20, 0xff, v28
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v22, v28, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v24, 0xff, v31
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v26, v31, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v40, 0xff, v30
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v42, v30, 16, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v31, 24, v33
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v29, v33, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v47, 24, v32
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v45, v32, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v51, 24, v35
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v49, v35, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v55, 24, v34
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v53, v34, 8, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v28, 0xff, v33
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v30, v33, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v44, 0xff, v32
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v46, v32, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v48, 0xff, v35
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v50, v35, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v52, 0xff, v34
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v54, v34, 16, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v35, 24, v37
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v33, v37, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v59, 24, v36
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v57, v36, 8, 8
-; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v63, 24, v39
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v61, v39, 8, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v32, 0xff, v37
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v34, v37, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v56, 0xff, v36
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v58, v36, 16, 8
-; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v60, 0xff, v39
-; GCN-NOHSA-VI-NEXT: v_bfe_u32 v62, v39, 16, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v19, 24, v21
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v17, v21, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v35, 24, v20
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v33, v20, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v39, 24, v23
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v37, v23, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v43, 24, v22
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v41, v22, 8, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v16, 0xff, v21
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v18, v21, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v32, 0xff, v20
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v34, v20, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v36, 0xff, v23
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v38, v23, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v40, 0xff, v22
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v42, v22, 16, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v23, 24, v25
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v21, v25, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v47, 24, v24
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v45, v24, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v51, 24, v27
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v49, v27, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v55, 24, v26
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v53, v26, 8, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v20, 0xff, v25
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v22, v25, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v44, 0xff, v24
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v46, v24, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v48, 0xff, v27
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v50, v27, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v52, 0xff, v26
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v54, v26, 16, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v27, 24, v29
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v25, v29, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v59, 24, v28
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v57, v28, 8, 8
+; GCN-NOHSA-VI-NEXT: v_lshrrev_b32_e32 v63, 24, v31
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v61, v31, 8, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v24, 0xff, v29
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v26, v29, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v56, 0xff, v28
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v58, v28, 16, 8
+; GCN-NOHSA-VI-NEXT: v_and_b32_e32 v60, 0xff, v31
+; GCN-NOHSA-VI-NEXT: v_bfe_u32 v62, v31, 16, 8
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[60:63], off, s[0:3], 0 offset:240
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[56:59], off, s[0:3], 0 offset:192
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:208
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:208
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[52:55], off, s[0:3], 0 offset:160
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[48:51], off, s[0:3], 0 offset:176
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[44:47], off, s[0:3], 0 offset:128
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:144
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:144
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[40:43], off, s[0:3], 0 offset:96
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:112
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:64
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[36:39], off, s[0:3], 0 offset:112
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:64
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:80
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:32
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
@@ -4065,9 +4065,9 @@ define amdgpu_kernel void @global_sextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-SI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-SI-NEXT: s_mov_b32 s11, s3
; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[10:13], off, s[8:11], 0
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[26:29], off, s[8:11], 0 offset:16
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[30:33], off, s[8:11], 0 offset:32
-; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[34:37], off, s[8:11], 0 offset:48
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[14:17], off, s[8:11], 0 offset:16
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[18:21], off, s[8:11], 0 offset:32
+; GCN-NOHSA-SI-NEXT: buffer_load_dwordx4 v[22:25], off, s[8:11], 0 offset:48
; GCN-NOHSA-SI-NEXT: s_addc_u32 s13, s13, 0
; GCN-NOHSA-SI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-SI-NEXT: s_mov_b32 s1, s5
@@ -4081,10 +4081,10 @@ define amdgpu_kernel void @global_sextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-SI-NEXT: buffer_store_dword v2, off, s[12:15], 0 offset:8 ; 4-byte Folded Spill
; GCN-NOHSA-SI-NEXT: buffer_store_dword v3, off, s[12:15], 0 offset:12 ; 4-byte Folded Spill
; GCN-NOHSA-SI-NEXT: s_waitcnt vmcnt(4) expcnt(0)
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v3, 24, v36
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v2, v36, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v1, v36, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v0, v36, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v3, 24, v24
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v2, v24, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v1, v24, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v0, v24, 0, 8
; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v7, 24, v10
; GCN-NOHSA-SI-NEXT: v_bfe_i32 v6, v10, 16, 8
; GCN-NOHSA-SI-NEXT: v_bfe_i32 v5, v10, 8, 8
@@ -4093,67 +4093,67 @@ define amdgpu_kernel void @global_sextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-SI-NEXT: v_bfe_i32 v10, v13, 16, 8
; GCN-NOHSA-SI-NEXT: v_bfe_i32 v9, v13, 8, 8
; GCN-NOHSA-SI-NEXT: v_bfe_i32 v8, v13, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v15, 24, v12
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v14, v12, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v13, v12, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v12, v12, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v19, 24, v27
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v18, v27, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v17, v27, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v16, v27, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v23, 24, v26
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v22, v26, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v21, v26, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v20, v26, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v27, 24, v29
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v26, v29, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v25, v29, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v24, v29, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v41, 24, v28
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v40, v28, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v39, v28, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v38, v28, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v45, 24, v31
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v44, v31, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v43, v31, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v42, v31, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v49, 24, v30
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v48, v30, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v47, v30, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v46, v30, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v31, 24, v33
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v30, v33, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v29, v33, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v28, v33, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v53, 24, v32
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v52, v32, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v51, v32, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v50, v32, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v57, 24, v35
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v56, v35, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v55, v35, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v54, v35, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v61, 24, v34
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v60, v34, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v59, v34, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v58, v34, 0, 8
-; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v35, 24, v37
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v34, v37, 16, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v33, v37, 8, 8
-; GCN-NOHSA-SI-NEXT: v_bfe_i32 v32, v37, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v29, 24, v12
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v28, v12, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v27, v12, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v26, v12, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v33, 24, v15
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v32, v15, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v31, v15, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v30, v15, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v37, 24, v14
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v36, v14, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v35, v14, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v34, v14, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v15, 24, v17
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v14, v17, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v13, v17, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v12, v17, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v41, 24, v16
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v40, v16, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v39, v16, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v38, v16, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v45, 24, v19
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v44, v19, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v43, v19, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v42, v19, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v49, 24, v18
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v48, v18, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v47, v18, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v46, v18, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v19, 24, v21
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v18, v21, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v17, v21, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v16, v21, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v53, 24, v20
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v52, v20, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v51, v20, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v50, v20, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v57, 24, v23
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v56, v23, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v55, v23, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v54, v23, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v61, 24, v22
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v60, v22, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v59, v22, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v58, v22, 0, 8
+; GCN-NOHSA-SI-NEXT: v_ashrrev_i32_e32 v23, 24, v25
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v22, v25, 16, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v21, v25, 8, 8
+; GCN-NOHSA-SI-NEXT: v_bfe_i32 v20, v25, 0, 8
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:240
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:240
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[58:61], off, s[0:3], 0 offset:192
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[54:57], off, s[0:3], 0 offset:208
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[50:53], off, s[0:3], 0 offset:160
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:176
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:176
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[46:49], off, s[0:3], 0 offset:128
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[42:45], off, s[0:3], 0 offset:144
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[38:41], off, s[0:3], 0 offset:96
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:112
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:64
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:80
-; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:32
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:112
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[34:37], off, s[0:3], 0 offset:64
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[30:33], off, s[0:3], 0 offset:80
+; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[26:29], off, s[0:3], 0 offset:32
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
; GCN-NOHSA-SI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0
; GCN-NOHSA-SI-NEXT: buffer_load_dword v0, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -4353,9 +4353,9 @@ define amdgpu_kernel void @global_sextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-VI-NEXT: s_mov_b32 s9, s7
; GCN-NOHSA-VI-NEXT: s_mov_b32 s11, s3
; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[10:13], off, s[8:11], 0
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[26:29], off, s[8:11], 0 offset:16
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[30:33], off, s[8:11], 0 offset:32
-; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[34:37], off, s[8:11], 0 offset:48
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[14:17], off, s[8:11], 0 offset:16
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[18:21], off, s[8:11], 0 offset:32
+; GCN-NOHSA-VI-NEXT: buffer_load_dwordx4 v[22:25], off, s[8:11], 0 offset:48
; GCN-NOHSA-VI-NEXT: s_addc_u32 s89, s89, 0
; GCN-NOHSA-VI-NEXT: s_mov_b32 s0, s4
; GCN-NOHSA-VI-NEXT: s_mov_b32 s1, s5
@@ -4369,10 +4369,10 @@ define amdgpu_kernel void @global_sextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-VI-NEXT: buffer_store_dword v2, off, s[88:91], 0 offset:8 ; 4-byte Folded Spill
; GCN-NOHSA-VI-NEXT: buffer_store_dword v3, off, s[88:91], 0 offset:12 ; 4-byte Folded Spill
; GCN-NOHSA-VI-NEXT: s_waitcnt vmcnt(4)
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v3, 24, v36
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v2, v36, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v1, v36, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v0, v36, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v3, 24, v24
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v2, v24, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v1, v24, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v0, v24, 0, 8
; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v7, 24, v10
; GCN-NOHSA-VI-NEXT: v_bfe_i32 v6, v10, 16, 8
; GCN-NOHSA-VI-NEXT: v_bfe_i32 v5, v10, 8, 8
@@ -4381,67 +4381,67 @@ define amdgpu_kernel void @global_sextload_v64i8_to_v64i32(ptr addrspace(1) %out
; GCN-NOHSA-VI-NEXT: v_bfe_i32 v10, v13, 16, 8
; GCN-NOHSA-VI-NEXT: v_bfe_i32 v9, v13, 8, 8
; GCN-NOHSA-VI-NEXT: v_bfe_i32 v8, v13, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v15, 24, v12
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v14, v12, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v13, v12, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v12, v12, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v19, 24, v27
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v18, v27, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v17, v27, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v16, v27, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v23, 24, v26
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v22, v26, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v21, v26, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v20, v26, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v27, 24, v29
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v26, v29, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v25, v29, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v24, v29, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v41, 24, v28
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v40, v28, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v39, v28, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v38, v28, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v45, 24, v31
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v44, v31, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v43, v31, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v42, v31, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v49, 24, v30
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v48, v30, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v47, v30, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v46, v30, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v31, 24, v33
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v30, v33, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v29, v33, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v28, v33, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v53, 24, v32
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v52, v32, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v51, v32, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v50, v32, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v57, 24, v35
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v56, v35, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v55, v35, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v54, v35, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v61, 24, v34
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v60, v34, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v59, v34, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v58, v34, 0, 8
-; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v35, 24, v37
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v34, v37, 16, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v33, v37, 8, 8
-; GCN-NOHSA-VI-NEXT: v_bfe_i32 v32, v37, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v29, 24, v12
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v28, v12, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v27, v12, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v26, v12, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v33, 24, v15
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v32, v15, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v31, v15, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v30, v15, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v37, 24, v14
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v36, v14, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v35, v14, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v34, v14, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v15, 24, v17
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v14, v17, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v13, v17, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v12, v17, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v41, 24, v16
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v40, v16, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v39, v16, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v38, v16, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v45, 24, v19
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v44, v19, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v43, v19, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v42, v19, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v49, 24, v18
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v48, v18, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v47, v18, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v46, v18, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v19, 24, v21
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v18, v21, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v17, v21, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v16, v21, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v53, 24, v20
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v52, v20, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v51, v20, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v50, v20, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v57, 24, v23
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v56, v23, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v55, v23, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v54, v23, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v61, 24, v22
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v60, v22, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v59, v22, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v58, v22, 0, 8
+; GCN-NOHSA-VI-NEXT: v_ashrrev_i32_e32 v23, 24, v25
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v22, v25, 16, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v21, v25, 8, 8
+; GCN-NOHSA-VI-NEXT: v_bfe_i32 v20, v25, 0, 8
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[0:3], off, s[0:3], 0 offset:224
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[32:35], off, s[0:3], 0 offset:240
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:240
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[58:61], off, s[0:3], 0 offset:192
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[54:57], off, s[0:3], 0 offset:208
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[50:53], off, s[0:3], 0 offset:160
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[28:31], off, s[0:3], 0 offset:176
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:176
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[46:49], off, s[0:3], 0 offset:128
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[42:45], off, s[0:3], 0 offset:144
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[38:41], off, s[0:3], 0 offset:96
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[24:27], off, s[0:3], 0 offset:112
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[20:23], off, s[0:3], 0 offset:64
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[16:19], off, s[0:3], 0 offset:80
-; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:32
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[12:15], off, s[0:3], 0 offset:112
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[34:37], off, s[0:3], 0 offset:64
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[30:33], off, s[0:3], 0 offset:80
+; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[26:29], off, s[0:3], 0 offset:32
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[8:11], off, s[0:3], 0 offset:48
; GCN-NOHSA-VI-NEXT: buffer_store_dwordx4 v[4:7], off, s[0:3], 0
; GCN-NOHSA-VI-NEXT: buffer_load_dword v0, off, s[88:91], 0 ; 4-byte Folded Reload
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-debug.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-debug.mir
index 8bb4596041122..960a7c231932d 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-debug.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-debug.mir
@@ -21,7 +21,7 @@ machineFunctionInfo:
isEntryFunction: true
body: |
; DEBUG: Machine code for function sink_and_inc_idx_when_skipping_small_region_1: IsSSA, NoPHIs, TracksLiveness
- ; DEBUG: [PreRARemat] Retrying function scheduling with new min. occupancy of 9 from rematerializing (original was 9, target was 10)
+ ; DEBUG: [PreRARemat] Retrying function scheduling with new min. occupancy of 10 from rematerializing (original was 9, target was 10)
; DEBUG-NEXT: ********** MI Scheduling **********
; DEBUG-NEXT: sink_and_inc_idx_when_skipping_small_region_1:%bb.2
; DEBUG-NEXT: From: %24:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
@@ -168,8 +168,12 @@ machineFunctionInfo:
isEntryFunction: true
body: |
; DEBUG: Machine code for function test_occ_inc_revert_all_regions: IsSSA, NoPHIs, TracksLiveness
- ; DEBUG: [PreRARemat] Retrying function scheduling with new min. occupancy of 6 from rematerializing (original was 5, target was 6)
- ; DEBUG-NOT: [PreRARemat] ==== ROLLBACK ====
+ ; DEBUG: [PreRARemat] Retrying function scheduling with new min. occupancy of 7 from rematerializing (original was 7, target was 8)
+ ; DEBUG: [PreRARemat] Region 1 cannot meet occupancy target, interrupting re-scheduling in all regions
+ ; DEBUG: [PreRARemat] Reverting re-scheduling in region 0
+ ; DEBUG: [PreRARemat] Reverting re-scheduling in region 1
+ ; DEBUG-NOT: [PreRARemat] Reverting re-scheduling in region 3
+ ; DEBUG-NOT: [PreRARemat] Reverting re-scheduling in region 4
bb.0:
successors: %bb.1
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
index 8c9e4a5a26c83..4b58fcb51983c 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
@@ -189,16 +189,16 @@ body: |
; GFX908-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.1:
; GFX908-NEXT: successors: %bb.2(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.2:
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_4]], implicit [[V_CVT_I32_F64_e32_5]]
@@ -240,16 +240,16 @@ body: |
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.1:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.2(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.2:
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_4]], implicit [[V_CVT_I32_F64_e32_5]]
@@ -856,15 +856,15 @@ body: |
; GFX908-NEXT: [[V_CVT_I32_F64_e32_19:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 19, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.1:
; GFX908-NEXT: successors: %bb.2(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]]
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]], implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]], implicit [[V_CVT_I32_F64_e32_24]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.2:
; GFX908-NEXT: successors: %bb.3(0x80000000)
@@ -920,15 +920,15 @@ body: |
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_19:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 19, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.1:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.2(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]]
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]], implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]], implicit [[V_CVT_I32_F64_e32_24]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.2:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.3(0x80000000)
@@ -5254,18 +5254,18 @@ body: |
; GFX908-NEXT: [[V_CVT_I32_F64_e32_36:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 36, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_37:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 37, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_38:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 38, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_39:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 39, implicit $exec, implicit $mode
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.1:
; GFX908-NEXT: successors: %bb.2(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_40:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 40, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_40]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_39:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 40, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_39]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.2:
; GFX908-NEXT: successors: %bb.3(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_39]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_40:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 39, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_40]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.3:
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_10]]
@@ -5334,18 +5334,18 @@ body: |
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_36:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 36, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_37:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 37, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_38:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 38, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_39:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 39, implicit $exec, implicit $mode
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.1:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.2(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_40:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 40, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_40]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_39:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 40, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_39]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.2:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.3(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_39]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_40:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 39, implicit $exec, implicit $mode
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_40]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_10]]
@@ -5504,18 +5504,18 @@ body: |
; GFX908-NEXT: [[V_CVT_I32_F64_e32_44:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 44, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_45:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 45, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_46:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 46, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_47:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 47, implicit $exec, implicit $mode
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.1:
; GFX908-NEXT: successors: %bb.2(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_48:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 48, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_48]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_47:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 48, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_47]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.2:
; GFX908-NEXT: successors: %bb.3(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_47]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_48:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 47, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_48]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.3:
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_10]]
@@ -5596,18 +5596,18 @@ body: |
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_44:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 44, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_45:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 45, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_46:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 46, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_47:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 47, implicit $exec, implicit $mode
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.1:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.2(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_48:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 48, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_48]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_47:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 48, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_47]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.2:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.3(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_47]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_48:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 47, implicit $exec, implicit $mode
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_48]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_10]]
@@ -9411,17 +9411,17 @@ body: |
; GFX908-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.1:
; GFX908-NEXT: successors: %bb.2(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.2:
- ; GFX908-NEXT: DBG_VALUE [[V_CVT_I32_F64_e32_23]], 0, 0
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-NEXT: DBG_VALUE %23:vgpr_32, 0, 0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_4]], implicit [[V_CVT_I32_F64_e32_5]]
@@ -9463,17 +9463,17 @@ body: |
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 22, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.1:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.2(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.2:
- ; GFX908-GCNTRACKERS-NEXT: DBG_VALUE [[V_CVT_I32_F64_e32_23]], 0, 0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-GCNTRACKERS-NEXT: DBG_VALUE %23:vgpr_32, 0, 0
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_24:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_24]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_4]], implicit [[V_CVT_I32_F64_e32_5]]
@@ -9571,17 +9571,17 @@ body: |
; GFX908-NEXT: [[V_CVT_I32_F64_e32_19:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 19, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-NEXT: INLINEASM &"v_or_b32 $0, 0, $1", attdialect, regdef:VGPR_32, def %22, reguse:VGPR_32, [[V_CVT_I32_F64_e32_4]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.1:
; GFX908-NEXT: successors: %bb.2(0x80000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]]
; GFX908-NEXT: {{ $}}
; GFX908-NEXT: bb.2:
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]]
; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_4]], implicit [[V_CVT_I32_F64_e32_5]]
@@ -9622,17 +9622,17 @@ body: |
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_19:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 19, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_20:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 20, implicit $exec, implicit $mode, implicit-def $m0
; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_21:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 21, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
; GFX908-GCNTRACKERS-NEXT: INLINEASM &"v_or_b32 $0, 0, $1", attdialect, regdef:VGPR_32, def %22, reguse:VGPR_32, [[V_CVT_I32_F64_e32_4]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.1:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.2(0x80000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_22:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 24, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]]
; GFX908-GCNTRACKERS-NEXT: {{ $}}
; GFX908-GCNTRACKERS-NEXT: bb.2:
- ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_22]]
+ ; GFX908-GCNTRACKERS-NEXT: [[V_CVT_I32_F64_e32_23:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 23, implicit $exec, implicit $mode
+ ; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_23]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]]
; GFX908-GCNTRACKERS-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_4]], implicit [[V_CVT_I32_F64_e32_5]]
diff --git a/llvm/test/CodeGen/AMDGPU/promote-constOffset-to-imm.ll b/llvm/test/CodeGen/AMDGPU/promote-constOffset-to-imm.ll
index b113380874cd3..b177cf18c7947 100644
--- a/llvm/test/CodeGen/AMDGPU/promote-constOffset-to-imm.ll
+++ b/llvm/test/CodeGen/AMDGPU/promote-constOffset-to-imm.ll
@@ -370,19 +370,19 @@ define hidden amdgpu_kernel void @clmem_read(ptr addrspace(1) %buffer) {
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
; GFX8-NEXT: s_swappc_b64 s[30:31], s[4:5]
; GFX8-NEXT: v_lshlrev_b32_e32 v1, 17, v0
-; GFX8-NEXT: v_and_b32_e32 v10, 0xfe000000, v1
+; GFX8-NEXT: v_and_b32_e32 v6, 0xfe000000, v1
; GFX8-NEXT: v_mov_b32_e32 v1, 3
; GFX8-NEXT: v_lshlrev_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
-; GFX8-NEXT: v_or_b32_e32 v0, v10, v0
+; GFX8-NEXT: v_or_b32_e32 v0, v6, v0
; GFX8-NEXT: v_mov_b32_e32 v1, s35
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s34, v0
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: v_add_u32_e32 v0, vcc, 0x2800, v0
; GFX8-NEXT: s_movk_i32 s0, 0x2800
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-NEXT: v_mov_b32_e32 v6, 0
-; GFX8-NEXT: v_mov_b32_e32 v7, 0
-; GFX8-NEXT: v_mov_b32_e32 v11, 0x7f
+; GFX8-NEXT: v_mov_b32_e32 v4, 0
+; GFX8-NEXT: v_mov_b32_e32 v5, 0
+; GFX8-NEXT: v_mov_b32_e32 v7, 0x7f
; GFX8-NEXT: s_movk_i32 s1, 0x800
; GFX8-NEXT: s_movk_i32 s2, 0x1000
; GFX8-NEXT: s_movk_i32 s3, 0x1800
@@ -398,62 +398,62 @@ define hidden amdgpu_kernel void @clmem_read(ptr addrspace(1) %buffer) {
; GFX8-NEXT: ; => This Inner Loop Header: Depth=2
; GFX8-NEXT: v_add_u32_e32 v8, vcc, 0xffffd800, v2
; GFX8-NEXT: v_addc_u32_e32 v9, vcc, -1, v3, vcc
-; GFX8-NEXT: flat_load_dwordx2 v[4:5], v[2:3]
-; GFX8-NEXT: flat_load_dwordx2 v[14:15], v[8:9]
+; GFX8-NEXT: flat_load_dwordx2 v[10:11], v[2:3]
+; GFX8-NEXT: flat_load_dwordx2 v[8:9], v[8:9]
; GFX8-NEXT: v_add_u32_e32 v12, vcc, 0xffffe000, v2
; GFX8-NEXT: v_addc_u32_e32 v13, vcc, -1, v3, vcc
; GFX8-NEXT: flat_load_dwordx2 v[12:13], v[12:13]
-; GFX8-NEXT: v_add_u32_e32 v8, vcc, 0xffffe800, v2
-; GFX8-NEXT: v_addc_u32_e32 v9, vcc, -1, v3, vcc
-; GFX8-NEXT: flat_load_dwordx2 v[18:19], v[8:9]
+; GFX8-NEXT: v_add_u32_e32 v14, vcc, 0xffffe800, v2
+; GFX8-NEXT: v_addc_u32_e32 v15, vcc, -1, v3, vcc
+; GFX8-NEXT: flat_load_dwordx2 v[14:15], v[14:15]
; GFX8-NEXT: v_add_u32_e32 v16, vcc, 0xfffff000, v2
; GFX8-NEXT: v_addc_u32_e32 v17, vcc, -1, v3, vcc
-; GFX8-NEXT: v_add_u32_e32 v20, vcc, 0xfffff800, v2
-; GFX8-NEXT: v_addc_u32_e32 v21, vcc, -1, v3, vcc
-; GFX8-NEXT: flat_load_dwordx2 v[8:9], v[16:17]
-; GFX8-NEXT: v_add_u32_e32 v16, vcc, s1, v2
-; GFX8-NEXT: v_addc_u32_e32 v17, vcc, 0, v3, vcc
+; GFX8-NEXT: v_add_u32_e32 v18, vcc, 0xfffff800, v2
+; GFX8-NEXT: v_addc_u32_e32 v19, vcc, -1, v3, vcc
+; GFX8-NEXT: v_add_u32_e32 v20, vcc, s1, v2
+; GFX8-NEXT: v_addc_u32_e32 v21, vcc, 0, v3, vcc
+; GFX8-NEXT: flat_load_dwordx2 v[16:17], v[16:17]
; GFX8-NEXT: s_addk_i32 s5, 0x2000
; GFX8-NEXT: s_cmp_gt_u32 s5, 0x3fffff
; GFX8-NEXT: s_waitcnt vmcnt(3)
-; GFX8-NEXT: v_add_u32_e32 v22, vcc, v14, v6
-; GFX8-NEXT: v_addc_u32_e32 v23, vcc, v15, v7, vcc
-; GFX8-NEXT: v_add_u32_e32 v6, vcc, s2, v2
-; GFX8-NEXT: flat_load_dwordx2 v[14:15], v[20:21]
-; GFX8-NEXT: flat_load_dwordx2 v[16:17], v[16:17]
-; GFX8-NEXT: v_addc_u32_e32 v7, vcc, 0, v3, vcc
+; GFX8-NEXT: v_add_u32_e32 v22, vcc, v8, v4
+; GFX8-NEXT: v_addc_u32_e32 v23, vcc, v9, v5, vcc
+; GFX8-NEXT: v_add_u32_e32 v4, vcc, s2, v2
+; GFX8-NEXT: flat_load_dwordx2 v[8:9], v[18:19]
+; GFX8-NEXT: flat_load_dwordx2 v[18:19], v[20:21]
+; GFX8-NEXT: v_addc_u32_e32 v5, vcc, 0, v3, vcc
; GFX8-NEXT: v_add_u32_e32 v20, vcc, s3, v2
; GFX8-NEXT: v_addc_u32_e32 v21, vcc, 0, v3, vcc
; GFX8-NEXT: s_waitcnt vmcnt(4)
; GFX8-NEXT: v_add_u32_e32 v22, vcc, v12, v22
; GFX8-NEXT: v_addc_u32_e32 v23, vcc, v13, v23, vcc
; GFX8-NEXT: v_add_u32_e32 v12, vcc, s4, v2
-; GFX8-NEXT: flat_load_dwordx2 v[6:7], v[6:7]
+; GFX8-NEXT: flat_load_dwordx2 v[4:5], v[4:5]
; GFX8-NEXT: flat_load_dwordx2 v[20:21], v[20:21]
; GFX8-NEXT: v_addc_u32_e32 v13, vcc, 0, v3, vcc
; GFX8-NEXT: s_waitcnt vmcnt(5)
-; GFX8-NEXT: v_add_u32_e32 v22, vcc, v18, v22
-; GFX8-NEXT: v_addc_u32_e32 v23, vcc, v19, v23, vcc
-; GFX8-NEXT: v_add_u32_e32 v18, vcc, s0, v2
+; GFX8-NEXT: v_add_u32_e32 v22, vcc, v14, v22
+; GFX8-NEXT: v_addc_u32_e32 v23, vcc, v15, v23, vcc
+; GFX8-NEXT: v_add_u32_e32 v14, vcc, s0, v2
; GFX8-NEXT: flat_load_dwordx2 v[12:13], v[12:13]
-; GFX8-NEXT: v_addc_u32_e32 v19, vcc, 0, v3, vcc
-; GFX8-NEXT: flat_load_dwordx2 v[18:19], v[18:19]
+; GFX8-NEXT: v_addc_u32_e32 v15, vcc, 0, v3, vcc
+; GFX8-NEXT: flat_load_dwordx2 v[14:15], v[14:15]
; GFX8-NEXT: v_add_u32_e32 v2, vcc, 0x10000, v2
; GFX8-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
; GFX8-NEXT: s_waitcnt vmcnt(6)
-; GFX8-NEXT: v_add_u32_e32 v8, vcc, v8, v22
-; GFX8-NEXT: v_addc_u32_e32 v9, vcc, v9, v23, vcc
+; GFX8-NEXT: v_add_u32_e32 v16, vcc, v16, v22
+; GFX8-NEXT: v_addc_u32_e32 v17, vcc, v17, v23, vcc
; GFX8-NEXT: s_waitcnt vmcnt(5)
-; GFX8-NEXT: v_add_u32_e32 v8, vcc, v14, v8
-; GFX8-NEXT: v_addc_u32_e32 v9, vcc, v15, v9, vcc
-; GFX8-NEXT: v_add_u32_e32 v4, vcc, v4, v8
-; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v5, v9, vcc
+; GFX8-NEXT: v_add_u32_e32 v8, vcc, v8, v16
+; GFX8-NEXT: v_addc_u32_e32 v9, vcc, v9, v17, vcc
+; GFX8-NEXT: v_add_u32_e32 v8, vcc, v10, v8
+; GFX8-NEXT: v_addc_u32_e32 v9, vcc, v11, v9, vcc
; GFX8-NEXT: s_waitcnt vmcnt(4)
-; GFX8-NEXT: v_add_u32_e32 v4, vcc, v16, v4
-; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v17, v5, vcc
+; GFX8-NEXT: v_add_u32_e32 v8, vcc, v18, v8
+; GFX8-NEXT: v_addc_u32_e32 v9, vcc, v19, v9, vcc
; GFX8-NEXT: s_waitcnt vmcnt(3)
-; GFX8-NEXT: v_add_u32_e32 v4, vcc, v6, v4
-; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v7, v5, vcc
+; GFX8-NEXT: v_add_u32_e32 v4, vcc, v4, v8
+; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v5, v9, vcc
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_add_u32_e32 v4, vcc, v20, v4
; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v21, v5, vcc
@@ -461,19 +461,19 @@ define hidden amdgpu_kernel void @clmem_read(ptr addrspace(1) %buffer) {
; GFX8-NEXT: v_add_u32_e32 v4, vcc, v12, v4
; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v13, v5, vcc
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_add_u32_e32 v6, vcc, v18, v4
-; GFX8-NEXT: v_addc_u32_e32 v7, vcc, v19, v5, vcc
+; GFX8-NEXT: v_add_u32_e32 v4, vcc, v14, v4
+; GFX8-NEXT: v_addc_u32_e32 v5, vcc, v15, v5, vcc
; GFX8-NEXT: s_cbranch_scc0 .LBB1_2
; GFX8-NEXT: ; %bb.3: ; %while.cond.loopexit
; GFX8-NEXT: ; in Loop: Header=BB1_1 Depth=1
-; GFX8-NEXT: v_subrev_u32_e32 v11, vcc, 1, v11
+; GFX8-NEXT: v_subrev_u32_e32 v7, vcc, 1, v7
; GFX8-NEXT: s_and_b64 vcc, exec, vcc
; GFX8-NEXT: s_cbranch_vccz .LBB1_1
; GFX8-NEXT: ; %bb.4: ; %while.end
; GFX8-NEXT: v_mov_b32_e32 v1, s35
-; GFX8-NEXT: v_add_u32_e32 v0, vcc, s34, v10
+; GFX8-NEXT: v_add_u32_e32 v0, vcc, s34, v6
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
-; GFX8-NEXT: flat_store_dwordx2 v[0:1], v[6:7]
+; GFX8-NEXT: flat_store_dwordx2 v[0:1], v[4:5]
; GFX8-NEXT: s_endpgm
;
; GFX900-LABEL: clmem_read:
@@ -523,57 +523,59 @@ define hidden amdgpu_kernel void @clmem_read(ptr addrspace(1) %buffer) {
; GFX900-NEXT: ; => This Inner Loop Header: Depth=2
; GFX900-NEXT: v_add_co_u32_e32 v8, vcc, 0xffffe000, v2
; GFX900-NEXT: v_addc_co_u32_e32 v9, vcc, -1, v3, vcc
-; GFX900-NEXT: global_load_dwordx2 v[14:15], v[8:9], off offset:-2048
+; GFX900-NEXT: global_load_dwordx2 v[20:21], v[8:9], off offset:-2048
+; GFX900-NEXT: global_load_dwordx2 v[22:23], v[8:9], off
+; GFX900-NEXT: v_add_co_u32_e32 v18, vcc, s2, v2
+; GFX900-NEXT: v_add_co_u32_e64 v16, s[0:1], s3, v2
+; GFX900-NEXT: v_addc_co_u32_e32 v19, vcc, -1, v3, vcc
+; GFX900-NEXT: v_addc_co_u32_e64 v17, s[0:1], 0, v3, s[0:1]
+; GFX900-NEXT: global_load_dwordx2 v[8:9], v[18:19], off offset:-2048
+; GFX900-NEXT: global_load_dwordx2 v[24:25], v[16:17], off offset:2048
; GFX900-NEXT: global_load_dwordx2 v[10:11], v[2:3], off offset:-4096
; GFX900-NEXT: global_load_dwordx2 v[12:13], v[2:3], off offset:-2048
+; GFX900-NEXT: global_load_dwordx2 v[14:15], v[2:3], off
; GFX900-NEXT: s_addk_i32 s5, 0x2000
; GFX900-NEXT: s_cmp_gt_u32 s5, 0x3fffff
-; GFX900-NEXT: s_waitcnt vmcnt(2)
-; GFX900-NEXT: v_add_co_u32_e32 v16, vcc, v14, v4
-; GFX900-NEXT: v_addc_co_u32_e32 v17, vcc, v15, v5, vcc
-; GFX900-NEXT: global_load_dwordx2 v[4:5], v[2:3], off
-; GFX900-NEXT: global_load_dwordx2 v[14:15], v[8:9], off
-; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_add_co_u32_e32 v14, vcc, v14, v16
-; GFX900-NEXT: v_addc_co_u32_e32 v15, vcc, v15, v17, vcc
-; GFX900-NEXT: v_add_co_u32_e32 v8, vcc, s2, v2
-; GFX900-NEXT: v_addc_co_u32_e32 v9, vcc, -1, v3, vcc
-; GFX900-NEXT: global_load_dwordx2 v[8:9], v[8:9], off offset:-2048
-; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_add_co_u32_e32 v14, vcc, v8, v14
-; GFX900-NEXT: v_addc_co_u32_e32 v15, vcc, v9, v15, vcc
-; GFX900-NEXT: global_load_dwordx2 v[8:9], v[2:3], off offset:2048
-; GFX900-NEXT: v_add_co_u32_e32 v14, vcc, v10, v14
-; GFX900-NEXT: v_addc_co_u32_e32 v11, vcc, v11, v15, vcc
-; GFX900-NEXT: v_add_co_u32_e64 v14, s[0:1], v12, v14
-; GFX900-NEXT: v_addc_co_u32_e64 v15, s[0:1], v13, v11, s[0:1]
-; GFX900-NEXT: v_add_co_u32_e32 v10, vcc, s3, v2
-; GFX900-NEXT: v_add_co_u32_e64 v12, s[0:1], s4, v2
-; GFX900-NEXT: v_addc_co_u32_e32 v11, vcc, 0, v3, vcc
-; GFX900-NEXT: v_addc_co_u32_e64 v13, vcc, 0, v3, s[0:1]
-; GFX900-NEXT: v_add_co_u32_e32 v16, vcc, v4, v14
-; GFX900-NEXT: v_addc_co_u32_e32 v17, vcc, v5, v15, vcc
-; GFX900-NEXT: global_load_dwordx2 v[4:5], v[12:13], off offset:-4096
-; GFX900-NEXT: global_load_dwordx2 v[14:15], v[10:11], off offset:2048
-; GFX900-NEXT: s_waitcnt vmcnt(2)
-; GFX900-NEXT: v_add_co_u32_e32 v16, vcc, v8, v16
-; GFX900-NEXT: v_addc_co_u32_e32 v17, vcc, v9, v17, vcc
-; GFX900-NEXT: global_load_dwordx2 v[8:9], v[12:13], off
-; GFX900-NEXT: global_load_dwordx2 v[10:11], v[12:13], off offset:2048
+; GFX900-NEXT: s_waitcnt vmcnt(6)
+; GFX900-NEXT: v_add_co_u32_e32 v18, vcc, v20, v4
+; GFX900-NEXT: v_addc_co_u32_e32 v19, vcc, v21, v5, vcc
+; GFX900-NEXT: v_add_co_u32_e32 v16, vcc, s4, v2
+; GFX900-NEXT: global_load_dwordx2 v[4:5], v[2:3], off offset:2048
+; GFX900-NEXT: v_addc_co_u32_e32 v17, vcc, 0, v3, vcc
+; GFX900-NEXT: s_waitcnt vmcnt(6)
+; GFX900-NEXT: v_add_co_u32_e32 v20, vcc, v22, v18
+; GFX900-NEXT: v_addc_co_u32_e32 v21, vcc, v23, v19, vcc
+; GFX900-NEXT: global_load_dwordx2 v[18:19], v[16:17], off offset:-4096
+; GFX900-NEXT: s_waitcnt vmcnt(6)
+; GFX900-NEXT: v_add_co_u32_e32 v22, vcc, v8, v20
+; GFX900-NEXT: v_addc_co_u32_e32 v23, vcc, v9, v21, vcc
+; GFX900-NEXT: global_load_dwordx2 v[8:9], v[16:17], off
+; GFX900-NEXT: global_load_dwordx2 v[20:21], v[16:17], off offset:2048
; GFX900-NEXT: v_add_co_u32_e32 v2, vcc, 0x10000, v2
; GFX900-NEXT: v_addc_co_u32_e32 v3, vcc, 0, v3, vcc
+; GFX900-NEXT: s_waitcnt vmcnt(6)
+; GFX900-NEXT: v_add_co_u32_e32 v10, vcc, v10, v22
+; GFX900-NEXT: v_addc_co_u32_e32 v11, vcc, v11, v23, vcc
+; GFX900-NEXT: s_waitcnt vmcnt(5)
+; GFX900-NEXT: v_add_co_u32_e32 v10, vcc, v12, v10
+; GFX900-NEXT: v_addc_co_u32_e32 v11, vcc, v13, v11, vcc
+; GFX900-NEXT: s_waitcnt vmcnt(4)
+; GFX900-NEXT: v_add_co_u32_e32 v10, vcc, v14, v10
+; GFX900-NEXT: v_addc_co_u32_e32 v11, vcc, v15, v11, vcc
; GFX900-NEXT: s_waitcnt vmcnt(3)
-; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v4, v16
-; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v5, v17, vcc
+; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v4, v10
+; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v5, v11, vcc
; GFX900-NEXT: s_waitcnt vmcnt(2)
-; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v14, v4
-; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v15, v5, vcc
+; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v18, v4
+; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v19, v5, vcc
+; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v24, v4
+; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v25, v5, vcc
; GFX900-NEXT: s_waitcnt vmcnt(1)
; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v8, v4
; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v9, v5, vcc
; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v10, v4
-; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v11, v5, vcc
+; GFX900-NEXT: v_add_co_u32_e32 v4, vcc, v20, v4
+; GFX900-NEXT: v_addc_co_u32_e32 v5, vcc, v21, v5, vcc
; GFX900-NEXT: s_cbranch_scc0 .LBB1_2
; GFX900-NEXT: ; %bb.3: ; %while.cond.loopexit
; GFX900-NEXT: ; in Loop: Header=BB1_1 Depth=1
diff --git a/llvm/test/CodeGen/AMDGPU/regpressure_printer.mir b/llvm/test/CodeGen/AMDGPU/regpressure_printer.mir
index be5f45ff4fcea..3e5e4f4397f10 100644
--- a/llvm/test/CodeGen/AMDGPU/regpressure_printer.mir
+++ b/llvm/test/CodeGen/AMDGPU/regpressure_printer.mir
@@ -1,6 +1,9 @@
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 3
-# RUN: llc -mtriple=amdgpu9.00-amd-amdhsa --filetype=null -verify-machineinstrs --run-pass=amdgpu-print-rp %s -filetype=null 2>&1 | FileCheck %s --check-prefix=RP --check-prefix=RPU
-# RUN: llc -mtriple=amdgpu9.00-amd-amdhsa --filetype=null -verify-machineinstrs --run-pass=amdgpu-print-rp -amdgpu-print-rp-downward %s -filetype=null 2>&1 | FileCheck %s --check-prefix=RP --check-prefix=RPD
+# RUN: llc -mtriple=amdgpu9.00-amd-amdhsa --filetype=null -verify-machineinstrs --run-pass=amdgpu-print-rp %s -filetype=null 2>&1 | FileCheck %s --check-prefix=RP
+# RUN: llc -mtriple=amdgpu9.00-amd-amdhsa --filetype=null -verify-machineinstrs --run-pass=amdgpu-print-rp -amdgpu-print-rp-downward %s -filetype=null 2>&1 | FileCheck %s --check-prefix=RP
+
+# Both runs share the RP prefix: the upward and downward trackers are expected
+# to report the same pressure for every function below.
---
@@ -157,227 +160,116 @@ machineFunctionInfo:
isEntryFunction: true
waveLimiter: true
body: |
- ; RPU-LABEL: name: only_dbg_value_sched_region
- ; RPU: bb.0:
- ; RPU-NEXT: Live-in:
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 1 %0:vgpr_32 = COPY $vgpr0
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 3 %1:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 3
- ; RPU-NEXT: 0 5 %2:vreg_64 = GLOBAL_LOAD_DWORDX2 %1:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: 0 5
- ; RPU-NEXT: 0 6 %3:vgpr_32 = GLOBAL_LOAD_DWORD %1:vreg_64, 8, 0, implicit $exec
- ; RPU-NEXT: 0 6
- ; RPU-NEXT: 0 7 undef %4.sub1:vreg_64 = V_ADD_U32_e32 %0:vgpr_32, %0:vgpr_32, implicit $exec
- ; RPU-NEXT: 0 7
- ; RPU-NEXT: 0 8 %4.sub0:vreg_64 = V_MOV_B32_e32 111, implicit $exec
- ; RPU-NEXT: 0 8
- ; RPU-NEXT: 0 9 %5:vreg_64 = COPY %2:vreg_64
- ; RPU-NEXT: 0 9
- ; RPU-NEXT: 0 9 undef %6.sub0:vreg_64 = V_ADD_F32_e32 %1.sub0:vreg_64, %5.sub0:vreg_64, implicit $mode, implicit $exec
- ; RPU-NEXT: 0 8
- ; RPU-NEXT: 0 8 dead %6.sub1:vreg_64 = V_ADD_F32_e32 %1.sub1:vreg_64, %5.sub0:vreg_64, implicit $mode, implicit $exec
- ; RPU-NEXT: 0 7
- ; RPU-NEXT: 0 7 %7:vgpr_32 = GLOBAL_LOAD_DWORD %5:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: 0 6
- ; RPU-NEXT: 0 8 %8:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 7
- ; RPU-NEXT: 0 9 %9:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 9
- ; RPU-NEXT: 0 11 %10:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 11
- ; RPU-NEXT: 0 12 undef %11.sub1:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 12
- ; RPU-NEXT: 0 13 %12:vgpr_32 = IMPLICIT_DEF
- ; RPU-NEXT: 0 13
- ; RPU-NEXT: 0 14 %13:vgpr_32 = IMPLICIT_DEF
- ; RPU-NEXT: 0 14
- ; RPU-NEXT: 0 16 %14:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 16
- ; RPU-NEXT: 0 18 %15:vreg_64 = IMPLICIT_DEF
- ; RPU-NEXT: 0 18
- ; RPU-NEXT: 0 19 %16:vgpr_32 = IMPLICIT_DEF
- ; RPU-NEXT: 0 19
- ; RPU-NEXT: 0 20 %17:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
- ; RPU-NEXT: 0 20
- ; RPU-NEXT: 0 21 %18:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
- ; RPU-NEXT: 0 21
- ; RPU-NEXT: 0 21 undef %19.sub0:vreg_64 = V_ADD_F32_e32 %7:vgpr_32, %2.sub0:vreg_64, implicit $mode, implicit $exec
- ; RPU-NEXT: 0 20
- ; RPU-NEXT: 0 20 %19.sub1:vreg_64 = V_ADD_F32_e32 %3:vgpr_32, %3:vgpr_32, implicit $mode, implicit $exec
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: 0 20
- ; RPU-NEXT: 0 20 GLOBAL_STORE_DWORDX2 %19:vreg_64, %4:vreg_64, 32, 0, implicit $exec
- ; RPU-NEXT: 0 16
- ; RPU-NEXT: 0 16 %11.sub0:vreg_64 = GLOBAL_LOAD_DWORD %9:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: 0 15
- ; RPU-NEXT: 0 15 %8.sub0:vreg_64 = GLOBAL_LOAD_DWORD %10:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: 0 14
- ; RPU-NEXT: 0 14 dead %20:vgpr_32 = GLOBAL_LOAD_DWORD %11:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: 0 12
- ; RPU-NEXT: 0 12 dead %21:vgpr_32 = GLOBAL_LOAD_DWORD %14:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: 0 10
- ; RPU-NEXT: 0 11 dead %22:vgpr_32 = GLOBAL_LOAD_DWORD %15:vreg_64, 0, 0, implicit $exec
- ; RPU-NEXT: 0 10
- ; RPU-NEXT: 0 10 %23:vreg_64 = V_LSHLREV_B64_e64 2, %8:vreg_64, implicit $exec
- ; RPU-NEXT: 0 9
- ; RPU-NEXT: 0 9 S_NOP 0, implicit %13:vgpr_32, implicit %23.sub0:vreg_64, implicit %12:vgpr_32, implicit %17:vgpr_32
- ; RPU-NEXT: 0 5
- ; RPU-NEXT: 0 5 GLOBAL_STORE_DWORD %15:vreg_64, %18:vgpr_32, 0, 0, implicit $exec
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: bb.1:
- ; RPU-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: 0 2 S_SETREG_IMM32_B32 0, 1, implicit-def $mode, implicit $mode
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: 0 2 S_SETREG_IMM32_B32 0, 1, implicit-def $mode, implicit $mode
- ; RPU-NEXT: DBG_VALUE
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: Live-thr: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: bb.2:
- ; RPU-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: Live-thr: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: bb.3:
- ; RPU-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 2
- ; RPU-NEXT: 0 2 S_NOP 0, implicit %0:vgpr_32
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 S_NOP 0, implicit %16:vgpr_32
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 S_ENDPGM 0
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: Live-out:
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ;
- ; RPD-LABEL: name: only_dbg_value_sched_region
- ; RPD: bb.0:
- ; RPD-NEXT: Live-in:
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 1 %0:vgpr_32 = COPY $vgpr0
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 3 %1:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 3
- ; RPD-NEXT: 0 5 %2:vreg_64 = GLOBAL_LOAD_DWORDX2 %1:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: 0 5
- ; RPD-NEXT: 0 6 %3:vgpr_32 = GLOBAL_LOAD_DWORD %1:vreg_64, 8, 0, implicit $exec
- ; RPD-NEXT: 0 6
- ; RPD-NEXT: 0 7 undef %4.sub1:vreg_64 = V_ADD_U32_e32 %0:vgpr_32, %0:vgpr_32, implicit $exec
- ; RPD-NEXT: 0 7
- ; RPD-NEXT: 0 8 %4.sub0:vreg_64 = V_MOV_B32_e32 111, implicit $exec
- ; RPD-NEXT: 0 8
- ; RPD-NEXT: 0 10 %5:vreg_64 = COPY %2:vreg_64
- ; RPD-NEXT: 0 9
- ; RPD-NEXT: 0 10 undef %6.sub0:vreg_64 = V_ADD_F32_e32 %1.sub0:vreg_64, %5.sub0:vreg_64, implicit $mode, implicit $exec
- ; RPD-NEXT: 0 8
- ; RPD-NEXT: 0 9 dead %6.sub1:vreg_64 = V_ADD_F32_e32 %1.sub1:vreg_64, %5.sub0:vreg_64, implicit $mode, implicit $exec
- ; RPD-NEXT: 0 7
- ; RPD-NEXT: 0 8 %7:vgpr_32 = GLOBAL_LOAD_DWORD %5:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: 0 6
- ; RPD-NEXT: 0 8 %8:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 7
- ; RPD-NEXT: 0 9 %9:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 9
- ; RPD-NEXT: 0 11 %10:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 11
- ; RPD-NEXT: 0 12 undef %11.sub1:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 12
- ; RPD-NEXT: 0 13 %12:vgpr_32 = IMPLICIT_DEF
- ; RPD-NEXT: 0 13
- ; RPD-NEXT: 0 14 %13:vgpr_32 = IMPLICIT_DEF
- ; RPD-NEXT: 0 14
- ; RPD-NEXT: 0 16 %14:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 16
- ; RPD-NEXT: 0 18 %15:vreg_64 = IMPLICIT_DEF
- ; RPD-NEXT: 0 18
- ; RPD-NEXT: 0 19 %16:vgpr_32 = IMPLICIT_DEF
- ; RPD-NEXT: 0 19
- ; RPD-NEXT: 0 20 %17:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
- ; RPD-NEXT: 0 20
- ; RPD-NEXT: 0 21 %18:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
- ; RPD-NEXT: 0 21
- ; RPD-NEXT: 0 22 undef %19.sub0:vreg_64 = V_ADD_F32_e32 %7:vgpr_32, %2.sub0:vreg_64, implicit $mode, implicit $exec
- ; RPD-NEXT: 0 20
- ; RPD-NEXT: 0 21 %19.sub1:vreg_64 = V_ADD_F32_e32 %3:vgpr_32, %3:vgpr_32, implicit $mode, implicit $exec
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: 0 20
- ; RPD-NEXT: 0 20 GLOBAL_STORE_DWORDX2 %19:vreg_64, %4:vreg_64, 32, 0, implicit $exec
- ; RPD-NEXT: 0 16
- ; RPD-NEXT: 0 17 %11.sub0:vreg_64 = GLOBAL_LOAD_DWORD %9:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: 0 15
- ; RPD-NEXT: 0 16 %8.sub0:vreg_64 = GLOBAL_LOAD_DWORD %10:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: 0 14
- ; RPD-NEXT: 0 15 dead %20:vgpr_32 = GLOBAL_LOAD_DWORD %11:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: 0 12
- ; RPD-NEXT: 0 13 dead %21:vgpr_32 = GLOBAL_LOAD_DWORD %14:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: 0 10
- ; RPD-NEXT: 0 11 dead %22:vgpr_32 = GLOBAL_LOAD_DWORD %15:vreg_64, 0, 0, implicit $exec
- ; RPD-NEXT: 0 10
- ; RPD-NEXT: 0 12 %23:vreg_64 = V_LSHLREV_B64_e64 2, %8:vreg_64, implicit $exec
- ; RPD-NEXT: 0 9
- ; RPD-NEXT: 0 9 S_NOP 0, implicit %13:vgpr_32, implicit %23.sub0:vreg_64, implicit %12:vgpr_32, implicit %17:vgpr_32
- ; RPD-NEXT: 0 5
- ; RPD-NEXT: 0 5 GLOBAL_STORE_DWORD %15:vreg_64, %18:vgpr_32, 0, 0, implicit $exec
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: bb.1:
- ; RPD-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: 0 2 S_SETREG_IMM32_B32 0, 1, implicit-def $mode, implicit $mode
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: 0 2 S_SETREG_IMM32_B32 0, 1, implicit-def $mode, implicit $mode
- ; RPD-NEXT: DBG_VALUE
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: Live-thr: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: bb.2:
- ; RPD-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: Live-thr: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: bb.3:
- ; RPD-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 2
- ; RPD-NEXT: 0 2 S_NOP 0, implicit %0:vgpr_32
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 S_NOP 0, implicit %16:vgpr_32
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 S_ENDPGM 0
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: Live-out:
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
+ ; RP-LABEL: name: only_dbg_value_sched_region
+ ; RP: bb.0:
+ ; RP-NEXT: Live-in:
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 1 %0:vgpr_32 = COPY $vgpr0
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 3 %1:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 3
+ ; RP-NEXT: 0 5 %2:vreg_64 = GLOBAL_LOAD_DWORDX2 %1:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: 0 5
+ ; RP-NEXT: 0 6 %3:vgpr_32 = GLOBAL_LOAD_DWORD %1:vreg_64, 8, 0, implicit $exec
+ ; RP-NEXT: 0 6
+ ; RP-NEXT: 0 7 undef %4.sub1:vreg_64 = V_ADD_U32_e32 %0:vgpr_32, %0:vgpr_32, implicit $exec
+ ; RP-NEXT: 0 7
+ ; RP-NEXT: 0 8 %4.sub0:vreg_64 = V_MOV_B32_e32 111, implicit $exec
+ ; RP-NEXT: 0 8
+ ; RP-NEXT: 0 9 %5:vreg_64 = COPY %2:vreg_64
+ ; RP-NEXT: 0 9
+ ; RP-NEXT: 0 9 undef %6.sub0:vreg_64 = V_ADD_F32_e32 %1.sub0:vreg_64, %5.sub0:vreg_64, implicit $mode, implicit $exec
+ ; RP-NEXT: 0 8
+ ; RP-NEXT: 0 8 dead %6.sub1:vreg_64 = V_ADD_F32_e32 %1.sub1:vreg_64, %5.sub0:vreg_64, implicit $mode, implicit $exec
+ ; RP-NEXT: 0 7
+ ; RP-NEXT: 0 7 %7:vgpr_32 = GLOBAL_LOAD_DWORD %5:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: 0 6
+ ; RP-NEXT: 0 8 %8:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 7
+ ; RP-NEXT: 0 9 %9:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 9
+ ; RP-NEXT: 0 11 %10:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 11
+ ; RP-NEXT: 0 12 undef %11.sub1:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 12
+ ; RP-NEXT: 0 13 %12:vgpr_32 = IMPLICIT_DEF
+ ; RP-NEXT: 0 13
+ ; RP-NEXT: 0 14 %13:vgpr_32 = IMPLICIT_DEF
+ ; RP-NEXT: 0 14
+ ; RP-NEXT: 0 16 %14:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 16
+ ; RP-NEXT: 0 18 %15:vreg_64 = IMPLICIT_DEF
+ ; RP-NEXT: 0 18
+ ; RP-NEXT: 0 19 %16:vgpr_32 = IMPLICIT_DEF
+ ; RP-NEXT: 0 19
+ ; RP-NEXT: 0 20 %17:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
+ ; RP-NEXT: 0 20
+ ; RP-NEXT: 0 21 %18:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
+ ; RP-NEXT: 0 21
+ ; RP-NEXT: 0 21 undef %19.sub0:vreg_64 = V_ADD_F32_e32 %7:vgpr_32, %2.sub0:vreg_64, implicit $mode, implicit $exec
+ ; RP-NEXT: 0 20
+ ; RP-NEXT: 0 20 %19.sub1:vreg_64 = V_ADD_F32_e32 %3:vgpr_32, %3:vgpr_32, implicit $mode, implicit $exec
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: 0 20
+ ; RP-NEXT: 0 20 GLOBAL_STORE_DWORDX2 %19:vreg_64, %4:vreg_64, 32, 0, implicit $exec
+ ; RP-NEXT: 0 16
+ ; RP-NEXT: 0 16 %11.sub0:vreg_64 = GLOBAL_LOAD_DWORD %9:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: 0 15
+ ; RP-NEXT: 0 15 %8.sub0:vreg_64 = GLOBAL_LOAD_DWORD %10:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: 0 14
+ ; RP-NEXT: 0 14 dead %20:vgpr_32 = GLOBAL_LOAD_DWORD %11:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: 0 12
+ ; RP-NEXT: 0 12 dead %21:vgpr_32 = GLOBAL_LOAD_DWORD %14:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: 0 10
+ ; RP-NEXT: 0 11 dead %22:vgpr_32 = GLOBAL_LOAD_DWORD %15:vreg_64, 0, 0, implicit $exec
+ ; RP-NEXT: 0 10
+ ; RP-NEXT: 0 10 %23:vreg_64 = V_LSHLREV_B64_e64 2, %8:vreg_64, implicit $exec
+ ; RP-NEXT: 0 9
+ ; RP-NEXT: 0 9 S_NOP 0, implicit %13:vgpr_32, implicit %23.sub0:vreg_64, implicit %12:vgpr_32, implicit %17:vgpr_32
+ ; RP-NEXT: 0 5
+ ; RP-NEXT: 0 5 GLOBAL_STORE_DWORD %15:vreg_64, %18:vgpr_32, 0, 0, implicit $exec
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: bb.1:
+ ; RP-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: 0 2 S_SETREG_IMM32_B32 0, 1, implicit-def $mode, implicit $mode
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: 0 2 S_SETREG_IMM32_B32 0, 1, implicit-def $mode, implicit $mode
+ ; RP-NEXT: DBG_VALUE
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: Live-thr: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: bb.2:
+ ; RP-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: Live-out: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: Live-thr: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: bb.3:
+ ; RP-NEXT: Live-in: %0:0000000000000003 %16:0000000000000003
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 2
+ ; RP-NEXT: 0 2 S_NOP 0, implicit %0:vgpr_32
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 S_NOP 0, implicit %16:vgpr_32
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 S_ENDPGM 0
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: Live-out:
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
bb.0:
liveins: $vgpr0
@@ -459,33 +351,19 @@ name: test_not_early_clobber_trivial
tracksRegLiveness: true
body: |
bb.0:
- ; RPU-LABEL: name: test_not_early_clobber_trivial
- ; RPU: Live-in:
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 1 %0:vgpr_32 = V_MOV_B32_e32 42, implicit $exec
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 %1:vgpr_32 = V_MOV_B32_e32 %0:vgpr_32, implicit $exec
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 S_NOP 0, implicit %1:vgpr_32
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: Live-out:
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ;
- ; RPD-LABEL: name: test_not_early_clobber_trivial
- ; RPD: Live-in:
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 1 %0:vgpr_32 = V_MOV_B32_e32 42, implicit $exec
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 2 %1:vgpr_32 = V_MOV_B32_e32 %0:vgpr_32, implicit $exec
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 S_NOP 0, implicit %1:vgpr_32
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: Live-out:
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
+ ; RP-LABEL: name: test_not_early_clobber_trivial
+ ; RP: Live-in:
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 1 %0:vgpr_32 = V_MOV_B32_e32 42, implicit $exec
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 %1:vgpr_32 = V_MOV_B32_e32 %0:vgpr_32, implicit $exec
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 S_NOP 0, implicit %1:vgpr_32
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: Live-out:
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
%0:vgpr_32 = V_MOV_B32_e32 42, implicit $exec
%1:vgpr_32 = V_MOV_B32_e32 %0, implicit $exec
S_NOP 0, implicit %1
@@ -494,111 +372,58 @@ body: |
name: movrel
tracksRegLiveness: true
body: |
- ; RPU-LABEL: name: movrel
- ; RPU: bb.0:
- ; RPU-NEXT: Live-in:
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 $sgpr0 = COPY $sgpr1
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 $sgpr2_sgpr3 = S_GETPC_B64
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 $sgpr1 = COPY killed $sgpr3
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 $sgpr0_sgpr1_sgpr2_sgpr3 = S_LOAD_DWORDX4_IMM $sgpr0_sgpr1, 0, 0
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 $sgpr0 = S_BUFFER_LOAD_DWORD_IMM $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 1 undef %0.sub5:vreg_512 = V_MOV_B32_e32 5, implicit $exec
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 S_CMP_GT_U32 $sgpr0, 15, implicit-def $scc
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 S_CBRANCH_SCC1 %bb.2, implicit $scc
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 S_BRANCH %bb.1
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: Live-out: %0:0000000000000C00
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: bb.1:
- ; RPU-NEXT: Live-in:
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 1 undef %0.sub5:vreg_512 = V_MOV_B32_e32 5, implicit $exec
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 $m0 = S_MOV_B32 killed $sgpr0
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 16 %0:vreg_512 = V_INDIRECT_REG_WRITE_MOVREL_B32_V16 %0:vreg_512(tied-def 0), 42, 3, implicit $m0, implicit $exec
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: Live-out: %0:0000000000000C00
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: bb.2:
- ; RPU-NEXT: Live-in: %0:0000000000000C00
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 %1:vgpr_32 = V_CVT_F32_UBYTE0_e64 %0.sub5:vreg_512, 0, 0, implicit $exec
- ; RPU-NEXT: 0 1
- ; RPU-NEXT: 0 1 EXP_DONE 0, %1:vgpr_32, undef %2:vgpr_32, undef %3:vgpr_32, undef %4:vgpr_32, -1, 0, 1, implicit $exec
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 0 0 S_ENDPGM 0
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: Live-out:
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ;
- ; RPD-LABEL: name: movrel
- ; RPD: bb.0:
- ; RPD-NEXT: Live-in:
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 $sgpr0 = COPY $sgpr1
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 $sgpr2_sgpr3 = S_GETPC_B64
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 $sgpr1 = COPY killed $sgpr3
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 $sgpr0_sgpr1_sgpr2_sgpr3 = S_LOAD_DWORDX4_IMM $sgpr0_sgpr1, 0, 0
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 $sgpr0 = S_BUFFER_LOAD_DWORD_IMM $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 1 undef %0.sub5:vreg_512 = V_MOV_B32_e32 5, implicit $exec
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 S_CMP_GT_U32 $sgpr0, 15, implicit-def $scc
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 S_CBRANCH_SCC1 %bb.2, implicit $scc
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 S_BRANCH %bb.1
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: Live-out: %0:0000000000000C00
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: bb.1:
- ; RPD-NEXT: Live-in:
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 1 undef %0.sub5:vreg_512 = V_MOV_B32_e32 5, implicit $exec
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 $m0 = S_MOV_B32 killed $sgpr0
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 16 %0:vreg_512 = V_INDIRECT_REG_WRITE_MOVREL_B32_V16 %0:vreg_512(tied-def 0), 42, 3, implicit $m0, implicit $exec
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: Live-out: %0:0000000000000C00
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: bb.2:
- ; RPD-NEXT: Live-in: %0:0000000000000C00
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 2 %1:vgpr_32 = V_CVT_F32_UBYTE0_e64 %0.sub5:vreg_512, 0, 0, implicit $exec
- ; RPD-NEXT: 0 1
- ; RPD-NEXT: 0 1 EXP_DONE 0, %1:vgpr_32, undef %2:vgpr_32, undef %3:vgpr_32, undef %4:vgpr_32, -1, 0, 1, implicit $exec
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 0 0 S_ENDPGM 0
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: Live-out:
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
+ ; RP-LABEL: name: movrel
+ ; RP: bb.0:
+ ; RP-NEXT: Live-in:
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 $sgpr0 = COPY $sgpr1
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 $sgpr2_sgpr3 = S_GETPC_B64
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 $sgpr1 = COPY killed $sgpr3
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 $sgpr0_sgpr1_sgpr2_sgpr3 = S_LOAD_DWORDX4_IMM $sgpr0_sgpr1, 0, 0
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 $sgpr0 = S_BUFFER_LOAD_DWORD_IMM $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 1 undef %0.sub5:vreg_512 = V_MOV_B32_e32 5, implicit $exec
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 S_CMP_GT_U32 $sgpr0, 15, implicit-def $scc
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 S_CBRANCH_SCC1 %bb.2, implicit $scc
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 S_BRANCH %bb.1
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: Live-out: %0:0000000000000C00
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: bb.1:
+ ; RP-NEXT: Live-in:
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 1 undef %0.sub5:vreg_512 = V_MOV_B32_e32 5, implicit $exec
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 $m0 = S_MOV_B32 killed $sgpr0
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 16 %0:vreg_512 = V_INDIRECT_REG_WRITE_MOVREL_B32_V16 %0:vreg_512(tied-def 0), 42, 3, implicit $m0, implicit $exec
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: Live-out: %0:0000000000000C00
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: bb.2:
+ ; RP-NEXT: Live-in: %0:0000000000000C00
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 %1:vgpr_32 = V_CVT_F32_UBYTE0_e64 %0.sub5:vreg_512, 0, 0, implicit $exec
+ ; RP-NEXT: 0 1
+ ; RP-NEXT: 0 1 EXP_DONE 0, %1:vgpr_32, undef %2:vgpr_32, undef %3:vgpr_32, undef %4:vgpr_32, -1, 0, 1, implicit $exec
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 0 0 S_ENDPGM 0
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: Live-out:
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
bb.0:
liveins: $sgpr1
$sgpr0 = COPY $sgpr1
@@ -629,33 +454,19 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0_sgpr1_sgpr2_sgpr3
- ; RPU-LABEL: name: test_partially_used_def
- ; RPU: Live-in:
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 4 0 %0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
- ; RPU-NEXT: 4 0
- ; RPU-NEXT: 4 0 %1:sgpr_128 = COPY %0:sgpr_128
- ; RPU-NEXT: 1 0
- ; RPU-NEXT: 1 0 S_NOP 0, implicit %1.sub1:sgpr_128
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: Live-out:
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ;
- ; RPD-LABEL: name: test_partially_used_def
- ; RPD: Live-in:
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 4 0 %0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
- ; RPD-NEXT: 4 0
- ; RPD-NEXT: 8 0 %1:sgpr_128 = COPY %0:sgpr_128
- ; RPD-NEXT: 1 0
- ; RPD-NEXT: 1 0 S_NOP 0, implicit %1.sub1:sgpr_128
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: Live-out:
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
+ ; RP-LABEL: name: test_partially_used_def
+ ; RP: Live-in:
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 4 0 %0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
+ ; RP-NEXT: 4 0
+ ; RP-NEXT: 4 0 %1:sgpr_128 = COPY %0:sgpr_128
+ ; RP-NEXT: 1 0
+ ; RP-NEXT: 1 0 S_NOP 0, implicit %1.sub1:sgpr_128
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: Live-out:
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
%0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
%1:sgpr_128 = COPY %0:sgpr_128
S_NOP 0, implicit %1.sub1
@@ -689,33 +500,19 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0_sgpr1_sgpr2_sgpr3
- ; RPU-LABEL: name: test_partially_used_def_and_early_clobber_def
- ; RPU: Live-in:
- ; RPU-NEXT: SGPR VGPR
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: 4 0 %0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
- ; RPU-NEXT: 4 0
- ; RPU-NEXT: 16 0 %1:sgpr_128 = COPY %0:sgpr_128, implicit-def %2:sgpr_128, implicit-def early-clobber %3:sgpr_128, implicit-def dead early-clobber %4:sgpr_128
- ; RPU-NEXT: 6 0
- ; RPU-NEXT: 6 0 S_NOP 0, implicit %1.sub1:sgpr_128, implicit %2.sub0_sub1:sgpr_128, implicit %3.sub0_sub1_sub2:sgpr_128
- ; RPU-NEXT: 0 0
- ; RPU-NEXT: Live-out:
- ; RPU-NEXT: Live-thr:
- ; RPU-NEXT: 0 0
- ;
- ; RPD-LABEL: name: test_partially_used_def_and_early_clobber_def
- ; RPD: Live-in:
- ; RPD-NEXT: SGPR VGPR
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: 4 0 %0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
- ; RPD-NEXT: 4 0
- ; RPD-NEXT: 20 0 %1:sgpr_128 = COPY %0:sgpr_128, implicit-def %2:sgpr_128, implicit-def early-clobber %3:sgpr_128, implicit-def dead early-clobber %4:sgpr_128
- ; RPD-NEXT: 6 0
- ; RPD-NEXT: 6 0 S_NOP 0, implicit %1.sub1:sgpr_128, implicit %2.sub0_sub1:sgpr_128, implicit %3.sub0_sub1_sub2:sgpr_128
- ; RPD-NEXT: 0 0
- ; RPD-NEXT: Live-out:
- ; RPD-NEXT: Live-thr:
- ; RPD-NEXT: 0 0
+ ; RP-LABEL: name: test_partially_used_def_and_early_clobber_def
+ ; RP: Live-in:
+ ; RP-NEXT: SGPR VGPR
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: 4 0 %0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
+ ; RP-NEXT: 4 0
+ ; RP-NEXT: 16 0 %1:sgpr_128 = COPY %0:sgpr_128, implicit-def %2:sgpr_128, implicit-def early-clobber %3:sgpr_128, implicit-def dead early-clobber %4:sgpr_128
+ ; RP-NEXT: 6 0
+ ; RP-NEXT: 6 0 S_NOP 0, implicit %1.sub1:sgpr_128, implicit %2.sub0_sub1:sgpr_128, implicit %3.sub0_sub1_sub2:sgpr_128
+ ; RP-NEXT: 0 0
+ ; RP-NEXT: Live-out:
+ ; RP-NEXT: Live-thr:
+ ; RP-NEXT: 0 0
%0:sgpr_128 = COPY $sgpr0_sgpr1_sgpr2_sgpr3
%1:sgpr_128 = COPY %0:sgpr_128, implicit-def %2:sgpr_128, implicit-def early-clobber %3:sgpr_128, implicit-def early-clobber %4:sgpr_128
S_NOP 0, implicit %1.sub1, implicit %2.sub0_sub1, implicit %3.sub0_sub1_sub2
diff --git a/llvm/test/CodeGen/AMDGPU/rem_i128.ll b/llvm/test/CodeGen/AMDGPU/rem_i128.ll
index a52d709ca28cc..814020168ae2c 100644
--- a/llvm/test/CodeGen/AMDGPU/rem_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/rem_i128.ll
@@ -10,26 +10,26 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-LABEL: v_srem_i128_vv:
; GFX9: ; %bb.0: ; %_udiv-special-cases
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT: v_ashrrev_i32_e32 v18, 31, v3
-; GFX9-NEXT: v_xor_b32_e32 v0, v0, v18
-; GFX9-NEXT: v_xor_b32_e32 v1, v1, v18
-; GFX9-NEXT: v_sub_co_u32_e32 v0, vcc, v0, v18
-; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v18, vcc
-; GFX9-NEXT: v_xor_b32_e32 v2, v2, v18
-; GFX9-NEXT: v_xor_b32_e32 v3, v3, v18
-; GFX9-NEXT: v_subb_co_u32_e32 v2, vcc, v2, v18, vcc
+; GFX9-NEXT: v_ashrrev_i32_e32 v20, 31, v3
+; GFX9-NEXT: v_xor_b32_e32 v0, v0, v20
+; GFX9-NEXT: v_xor_b32_e32 v1, v1, v20
+; GFX9-NEXT: v_sub_co_u32_e32 v0, vcc, v0, v20
+; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v20, vcc
+; GFX9-NEXT: v_xor_b32_e32 v2, v2, v20
+; GFX9-NEXT: v_xor_b32_e32 v3, v3, v20
+; GFX9-NEXT: v_subb_co_u32_e32 v2, vcc, v2, v20, vcc
; GFX9-NEXT: v_ashrrev_i32_e32 v8, 31, v7
-; GFX9-NEXT: v_subb_co_u32_e32 v3, vcc, v3, v18, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v3, vcc, v3, v20, vcc
; GFX9-NEXT: v_xor_b32_e32 v4, v4, v8
; GFX9-NEXT: v_xor_b32_e32 v5, v5, v8
-; GFX9-NEXT: v_sub_co_u32_e32 v20, vcc, v4, v8
-; GFX9-NEXT: v_subb_co_u32_e32 v19, vcc, v5, v8, vcc
+; GFX9-NEXT: v_sub_co_u32_e32 v22, vcc, v4, v8
+; GFX9-NEXT: v_subb_co_u32_e32 v21, vcc, v5, v8, vcc
; GFX9-NEXT: v_xor_b32_e32 v4, v6, v8
; GFX9-NEXT: v_xor_b32_e32 v5, v7, v8
; GFX9-NEXT: v_subb_co_u32_e32 v4, vcc, v4, v8, vcc
; GFX9-NEXT: v_subb_co_u32_e32 v5, vcc, v5, v8, vcc
-; GFX9-NEXT: v_or_b32_e32 v7, v19, v5
-; GFX9-NEXT: v_or_b32_e32 v6, v20, v4
+; GFX9-NEXT: v_or_b32_e32 v7, v21, v5
+; GFX9-NEXT: v_or_b32_e32 v6, v22, v4
; GFX9-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[6:7]
; GFX9-NEXT: v_or_b32_e32 v7, v1, v3
; GFX9-NEXT: v_or_b32_e32 v6, v0, v2
@@ -38,9 +38,9 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: v_add_u32_e32 v6, 32, v6
; GFX9-NEXT: v_ffbh_u32_e32 v7, v5
; GFX9-NEXT: v_min_u32_e32 v6, v6, v7
-; GFX9-NEXT: v_ffbh_u32_e32 v7, v20
+; GFX9-NEXT: v_ffbh_u32_e32 v7, v22
; GFX9-NEXT: v_add_u32_e32 v7, 32, v7
-; GFX9-NEXT: v_ffbh_u32_e32 v8, v19
+; GFX9-NEXT: v_ffbh_u32_e32 v8, v21
; GFX9-NEXT: v_min_u32_e32 v7, v7, v8
; GFX9-NEXT: s_or_b64 s[4:5], vcc, s[4:5]
; GFX9-NEXT: v_add_co_u32_e32 v7, vcc, 64, v7
@@ -88,10 +88,10 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: s_and_saveexec_b64 s[6:7], s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB0_6
; GFX9-NEXT: ; %bb.1: ; %udiv-bb1
-; GFX9-NEXT: v_add_co_u32_e32 v21, vcc, 1, v8
-; GFX9-NEXT: v_addc_co_u32_e32 v22, vcc, 0, v9, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v23, vcc, 0, v10, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v24, vcc, 0, v11, vcc
+; GFX9-NEXT: v_add_co_u32_e32 v23, vcc, 1, v8
+; GFX9-NEXT: v_addc_co_u32_e32 v24, vcc, 0, v9, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v25, vcc, 0, v10, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v26, vcc, 0, v11, vcc
; GFX9-NEXT: v_sub_u32_e32 v11, 0x7f, v8
; GFX9-NEXT: v_sub_u32_e32 v9, 64, v11
; GFX9-NEXT: v_lshlrev_b64 v[6:7], v11, v[2:3]
@@ -116,71 +116,71 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: s_xor_b64 s[8:9], exec, s[4:5]
; GFX9-NEXT: s_cbranch_execz .LBB0_5
; GFX9-NEXT: ; %bb.2: ; %udiv-preheader
-; GFX9-NEXT: v_sub_u32_e32 v12, 64, v21
-; GFX9-NEXT: v_lshrrev_b64 v[10:11], v21, v[0:1]
+; GFX9-NEXT: v_sub_u32_e32 v12, 64, v23
+; GFX9-NEXT: v_lshrrev_b64 v[10:11], v23, v[0:1]
; GFX9-NEXT: v_lshlrev_b64 v[12:13], v12, v[2:3]
-; GFX9-NEXT: v_cmp_gt_u32_e32 vcc, 64, v21
+; GFX9-NEXT: v_cmp_gt_u32_e32 vcc, 64, v23
; GFX9-NEXT: v_or_b32_e32 v12, v10, v12
-; GFX9-NEXT: v_subrev_u32_e32 v10, 64, v21
+; GFX9-NEXT: v_subrev_u32_e32 v10, 64, v23
; GFX9-NEXT: v_or_b32_e32 v13, v11, v13
; GFX9-NEXT: v_lshrrev_b64 v[10:11], v10, v[2:3]
-; GFX9-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v21
+; GFX9-NEXT: v_cmp_eq_u32_e64 s[4:5], 0, v23
; GFX9-NEXT: v_cndmask_b32_e32 v11, v11, v13, vcc
; GFX9-NEXT: v_cndmask_b32_e64 v13, v11, v1, s[4:5]
; GFX9-NEXT: v_cndmask_b32_e32 v12, v10, v12, vcc
-; GFX9-NEXT: v_lshrrev_b64 v[10:11], v21, v[2:3]
+; GFX9-NEXT: v_lshrrev_b64 v[10:11], v23, v[2:3]
; GFX9-NEXT: v_cndmask_b32_e64 v12, v12, v0, s[4:5]
-; GFX9-NEXT: v_cndmask_b32_e32 v15, 0, v11, vcc
-; GFX9-NEXT: v_cndmask_b32_e32 v14, 0, v10, vcc
-; GFX9-NEXT: v_add_co_u32_e32 v25, vcc, -1, v20
-; GFX9-NEXT: v_addc_co_u32_e32 v26, vcc, -1, v19, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v27, vcc, -1, v4, vcc
-; GFX9-NEXT: v_addc_co_u32_e32 v28, vcc, -1, v5, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v17, 0, v11, vcc
+; GFX9-NEXT: v_cndmask_b32_e32 v16, 0, v10, vcc
+; GFX9-NEXT: v_add_co_u32_e32 v27, vcc, -1, v22
+; GFX9-NEXT: v_addc_co_u32_e32 v28, vcc, -1, v21, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v29, vcc, -1, v4, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v30, vcc, -1, v5, vcc
+; GFX9-NEXT: v_mov_b32_e32 v14, 0
+; GFX9-NEXT: v_mov_b32_e32 v15, 0
; GFX9-NEXT: s_mov_b64 s[4:5], 0
-; GFX9-NEXT: v_mov_b32_e32 v16, 0
-; GFX9-NEXT: v_mov_b32_e32 v17, 0
+; GFX9-NEXT: v_mov_b32_e32 v18, 0
+; GFX9-NEXT: v_mov_b32_e32 v19, 0
; GFX9-NEXT: v_mov_b32_e32 v11, 0
; GFX9-NEXT: .LBB0_3: ; %udiv-do-while
; GFX9-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-NEXT: v_lshlrev_b64 v[29:30], 1, v[8:9]
-; GFX9-NEXT: v_lshlrev_b64 v[14:15], 1, v[14:15]
-; GFX9-NEXT: v_or_b32_e32 v8, v16, v29
-; GFX9-NEXT: v_lshrrev_b32_e32 v16, 31, v13
+; GFX9-NEXT: v_lshlrev_b64 v[31:32], 1, v[8:9]
+; GFX9-NEXT: v_lshlrev_b64 v[16:17], 1, v[16:17]
+; GFX9-NEXT: v_or_b32_e32 v8, v18, v31
+; GFX9-NEXT: v_lshrrev_b32_e32 v18, 31, v13
; GFX9-NEXT: v_lshlrev_b64 v[12:13], 1, v[12:13]
-; GFX9-NEXT: v_or_b32_e32 v14, v14, v16
-; GFX9-NEXT: v_lshrrev_b32_e32 v16, 31, v7
-; GFX9-NEXT: v_or_b32_e32 v12, v12, v16
-; GFX9-NEXT: v_sub_co_u32_e32 v16, vcc, v25, v12
-; GFX9-NEXT: v_subb_co_u32_e32 v16, vcc, v26, v13, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v16, vcc, v27, v14, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v16, vcc, v28, v15, vcc
-; GFX9-NEXT: v_ashrrev_i32_e32 v29, 31, v16
-; GFX9-NEXT: v_and_b32_e32 v16, v29, v20
+; GFX9-NEXT: v_or_b32_e32 v16, v16, v18
+; GFX9-NEXT: v_lshrrev_b32_e32 v18, 31, v7
+; GFX9-NEXT: v_or_b32_e32 v12, v12, v18
+; GFX9-NEXT: v_sub_co_u32_e32 v18, vcc, v27, v12
+; GFX9-NEXT: v_subb_co_u32_e32 v18, vcc, v28, v13, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v18, vcc, v29, v16, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v18, vcc, v30, v17, vcc
+; GFX9-NEXT: v_ashrrev_i32_e32 v31, 31, v18
+; GFX9-NEXT: v_and_b32_e32 v18, v31, v22
; GFX9-NEXT: v_lshrrev_b32_e32 v10, 31, v9
-; GFX9-NEXT: v_or_b32_e32 v9, v17, v30
-; GFX9-NEXT: v_and_b32_e32 v17, v29, v19
-; GFX9-NEXT: v_sub_co_u32_e32 v12, vcc, v12, v16
-; GFX9-NEXT: v_subb_co_u32_e32 v13, vcc, v13, v17, vcc
-; GFX9-NEXT: v_and_b32_e32 v16, v29, v4
-; GFX9-NEXT: v_and_b32_e32 v17, v29, v5
-; GFX9-NEXT: v_subb_co_u32_e32 v14, vcc, v14, v16, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v15, vcc, v15, v17, vcc
-; GFX9-NEXT: v_add_co_u32_e32 v21, vcc, -1, v21
-; GFX9-NEXT: v_addc_co_u32_e32 v22, vcc, -1, v22, vcc
-; GFX9-NEXT: v_lshlrev_b64 v[6:7], 1, v[6:7]
-; GFX9-NEXT: v_addc_co_u32_e32 v23, vcc, -1, v23, vcc
-; GFX9-NEXT: v_mov_b32_e32 v16, 0
-; GFX9-NEXT: v_mov_b32_e32 v17, 0
+; GFX9-NEXT: v_or_b32_e32 v9, v19, v32
+; GFX9-NEXT: v_and_b32_e32 v19, v31, v21
+; GFX9-NEXT: v_sub_co_u32_e32 v12, vcc, v12, v18
+; GFX9-NEXT: v_subb_co_u32_e32 v13, vcc, v13, v19, vcc
+; GFX9-NEXT: v_and_b32_e32 v18, v31, v4
+; GFX9-NEXT: v_and_b32_e32 v19, v31, v5
+; GFX9-NEXT: v_subb_co_u32_e32 v16, vcc, v16, v18, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v17, vcc, v17, v19, vcc
+; GFX9-NEXT: v_add_co_u32_e32 v23, vcc, -1, v23
; GFX9-NEXT: v_addc_co_u32_e32 v24, vcc, -1, v24, vcc
-; GFX9-NEXT: v_or3_b32 v6, v6, v10, v16
-; GFX9-NEXT: v_or3_b32 v7, v7, 0, v17
-; GFX9-NEXT: v_or_b32_e32 v16, v21, v23
-; GFX9-NEXT: v_or_b32_e32 v17, v22, v24
-; GFX9-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[16:17]
-; GFX9-NEXT: v_and_b32_e32 v10, 1, v29
+; GFX9-NEXT: v_addc_co_u32_e32 v25, vcc, -1, v25, vcc
+; GFX9-NEXT: v_addc_co_u32_e32 v26, vcc, -1, v26, vcc
+; GFX9-NEXT: v_lshlrev_b64 v[6:7], 1, v[6:7]
+; GFX9-NEXT: v_or_b32_e32 v18, v23, v25
+; GFX9-NEXT: v_or_b32_e32 v19, v24, v26
+; GFX9-NEXT: v_cmp_eq_u64_e32 vcc, 0, v[18:19]
+; GFX9-NEXT: v_or3_b32 v6, v6, v10, v14
+; GFX9-NEXT: v_and_b32_e32 v10, 1, v31
+; GFX9-NEXT: v_or3_b32 v7, v7, 0, v15
; GFX9-NEXT: s_or_b64 s[4:5], vcc, s[4:5]
-; GFX9-NEXT: v_mov_b32_e32 v17, v11
-; GFX9-NEXT: v_mov_b32_e32 v16, v10
+; GFX9-NEXT: v_mov_b32_e32 v19, v11
+; GFX9-NEXT: v_mov_b32_e32 v18, v10
; GFX9-NEXT: s_andn2_b64 exec, exec, s[4:5]
; GFX9-NEXT: s_cbranch_execnz .LBB0_3
; GFX9-NEXT: ; %bb.4: ; %Flow
@@ -195,21 +195,21 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: v_or_b32_e32 v13, v10, v8
; GFX9-NEXT: .LBB0_6: ; %Flow3
; GFX9-NEXT: s_or_b64 exec, exec, s[6:7]
-; GFX9-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v20, v13, 0
+; GFX9-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v22, v13, 0
; GFX9-NEXT: v_mov_b32_e32 v10, 0
; GFX9-NEXT: v_mul_lo_u32 v11, v12, v4
; GFX9-NEXT: v_mul_lo_u32 v16, v13, v5
-; GFX9-NEXT: v_mad_u64_u32 v[14:15], s[4:5], v19, v13, v[9:10]
+; GFX9-NEXT: v_mad_u64_u32 v[14:15], s[4:5], v21, v13, v[9:10]
; GFX9-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v13, v4, 0
-; GFX9-NEXT: v_mul_lo_u32 v13, v6, v19
-; GFX9-NEXT: v_mul_lo_u32 v17, v7, v20
+; GFX9-NEXT: v_mul_lo_u32 v13, v6, v21
+; GFX9-NEXT: v_mul_lo_u32 v17, v7, v22
; GFX9-NEXT: v_mov_b32_e32 v9, v14
-; GFX9-NEXT: v_mad_u64_u32 v[9:10], s[4:5], v20, v12, v[9:10]
+; GFX9-NEXT: v_mad_u64_u32 v[9:10], s[4:5], v22, v12, v[9:10]
; GFX9-NEXT: v_add3_u32 v5, v5, v16, v11
-; GFX9-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v6, v20, v[4:5]
+; GFX9-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v6, v22, v[4:5]
; GFX9-NEXT: v_add_co_u32_e32 v6, vcc, v15, v10
; GFX9-NEXT: v_addc_co_u32_e64 v7, s[4:5], 0, 0, vcc
-; GFX9-NEXT: v_mad_u64_u32 v[6:7], s[4:5], v19, v12, v[6:7]
+; GFX9-NEXT: v_mad_u64_u32 v[6:7], s[4:5], v21, v12, v[6:7]
; GFX9-NEXT: v_add3_u32 v5, v17, v5, v13
; GFX9-NEXT: v_add_co_u32_e32 v4, vcc, v6, v4
; GFX9-NEXT: v_addc_co_u32_e32 v5, vcc, v7, v5, vcc
@@ -217,14 +217,14 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v9, vcc
; GFX9-NEXT: v_subb_co_u32_e32 v2, vcc, v2, v4, vcc
; GFX9-NEXT: v_subb_co_u32_e32 v3, vcc, v3, v5, vcc
-; GFX9-NEXT: v_xor_b32_e32 v0, v0, v18
-; GFX9-NEXT: v_xor_b32_e32 v1, v1, v18
-; GFX9-NEXT: v_sub_co_u32_e32 v0, vcc, v0, v18
-; GFX9-NEXT: v_xor_b32_e32 v2, v2, v18
-; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v18, vcc
-; GFX9-NEXT: v_xor_b32_e32 v3, v3, v18
-; GFX9-NEXT: v_subb_co_u32_e32 v2, vcc, v2, v18, vcc
-; GFX9-NEXT: v_subb_co_u32_e32 v3, vcc, v3, v18, vcc
+; GFX9-NEXT: v_xor_b32_e32 v0, v0, v20
+; GFX9-NEXT: v_xor_b32_e32 v1, v1, v20
+; GFX9-NEXT: v_sub_co_u32_e32 v0, vcc, v0, v20
+; GFX9-NEXT: v_xor_b32_e32 v2, v2, v20
+; GFX9-NEXT: v_subb_co_u32_e32 v1, vcc, v1, v20, vcc
+; GFX9-NEXT: v_xor_b32_e32 v3, v3, v20
+; GFX9-NEXT: v_subb_co_u32_e32 v2, vcc, v2, v20, vcc
+; GFX9-NEXT: v_subb_co_u32_e32 v3, vcc, v3, v20, vcc
; GFX9-NEXT: s_setpc_b64 s[30:31]
;
; GFX9-O0-LABEL: v_srem_i128_vv:
diff --git a/llvm/test/CodeGen/AMDGPU/rewrite-vgpr-mfma-to-agpr-spill-multi-store-codegen.ll b/llvm/test/CodeGen/AMDGPU/rewrite-vgpr-mfma-to-agpr-spill-multi-store-codegen.ll
index 987b827a48664..c1df73c044bb4 100644
--- a/llvm/test/CodeGen/AMDGPU/rewrite-vgpr-mfma-to-agpr-spill-multi-store-codegen.ll
+++ b/llvm/test/CodeGen/AMDGPU/rewrite-vgpr-mfma-to-agpr-spill-multi-store-codegen.ll
@@ -11,37 +11,68 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: s_mov_b32 s0, 0
; CHECK-NEXT: v_mov_b32_e32 v3, 0
; CHECK-NEXT: s_mov_b32 s1, s0
-; CHECK-NEXT: v_mov_b32_e32 v28, 0x3c003c00
-; CHECK-NEXT: v_pk_mov_b32 v[30:31], s[0:1], s[0:1] op_sel:[0,1]
+; CHECK-NEXT: v_mov_b32_e32 v132, 0x3c003c00
+; CHECK-NEXT: v_pk_mov_b32 v[134:135], s[0:1], s[0:1] op_sel:[0,1]
; CHECK-NEXT: v_mov_b32_e32 v0, 0x7fc00000
-; CHECK-NEXT: v_mov_b32_e32 v213, 1.0
-; CHECK-NEXT: v_mov_b32_e32 v29, v28
+; CHECK-NEXT: v_accvgpr_write_b32 a1, 1.0
+; CHECK-NEXT: v_mov_b32_e32 v133, v132
; CHECK-NEXT: s_and_b64 vcc, exec, -1
; CHECK-NEXT: v_mov_b32_e32 v2, v3
-; CHECK-NEXT: v_mov_b32_e32 v70, v3
+; CHECK-NEXT: v_mov_b32_e32 v22, v3
+; CHECK-NEXT: v_mov_b32_e32 v18, v3
+; CHECK-NEXT: v_mov_b32_e32 v24, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a17, v3
; CHECK-NEXT: v_mov_b32_e32 v20, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a64, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a1, v3
-; CHECK-NEXT: v_mov_b32_e32 v86, v3
-; CHECK-NEXT: v_mov_b32_e32 v162, v3
+; CHECK-NEXT: v_mov_b32_e32 v180, v3
; CHECK-NEXT: v_mov_b32_e32 v50, v3
-; CHECK-NEXT: v_mov_b32_e32 v212, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a0, v3
; CHECK-NEXT: v_mov_b32_e32 v67, v3
; CHECK-NEXT: v_mov_b32_e32 v66, v3
-; CHECK-NEXT: v_mov_b32_e32 v196, v3
-; CHECK-NEXT: v_mov_b32_e32 v146, v3
-; CHECK-NEXT: v_mov_b32_e32 v32, v3
+; CHECK-NEXT: v_mov_b32_e32 v82, v3
+; CHECK-NEXT: v_mov_b32_e32 v98, v3
+; CHECK-NEXT: v_mov_b32_e32 v226, v3
; CHECK-NEXT: v_mov_b32_e32 v243, v3
; CHECK-NEXT: v_mov_b32_e32 v114, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a17, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a16, v3
-; CHECK-NEXT: v_mov_b32_e32 v145, v3
-; CHECK-NEXT: v_mov_b32_e32 v34, v3
-; CHECK-NEXT: v_mov_b32_e32 v35, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a33, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a32, v3
+; CHECK-NEXT: v_mov_b32_e32 v161, v3
+; CHECK-NEXT: v_mov_b32_e32 v162, v3
+; CHECK-NEXT: v_mov_b32_e32 v163, v3
; CHECK-NEXT: .LBB0_1: ; %do.body
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
+; CHECK-NEXT: v_accvgpr_mov_b32 a2, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a3, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a4, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a5, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a6, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a7, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a8, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a9, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a10, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a11, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a12, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a13, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a14, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a15, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a49, a15
+; CHECK-NEXT: v_accvgpr_mov_b32 a48, a14
+; CHECK-NEXT: v_accvgpr_mov_b32 a47, a13
+; CHECK-NEXT: v_accvgpr_mov_b32 a46, a12
+; CHECK-NEXT: v_accvgpr_mov_b32 a45, a11
+; CHECK-NEXT: v_accvgpr_mov_b32 a44, a10
+; CHECK-NEXT: v_accvgpr_mov_b32 a43, a9
+; CHECK-NEXT: v_accvgpr_mov_b32 a42, a8
+; CHECK-NEXT: v_accvgpr_mov_b32 a41, a7
+; CHECK-NEXT: v_accvgpr_mov_b32 a40, a6
+; CHECK-NEXT: v_accvgpr_mov_b32 a39, a5
+; CHECK-NEXT: v_accvgpr_mov_b32 a38, a4
+; CHECK-NEXT: v_accvgpr_mov_b32 a37, a3
+; CHECK-NEXT: v_accvgpr_mov_b32 a36, a2
+; CHECK-NEXT: v_accvgpr_mov_b32 a35, a1
+; CHECK-NEXT: v_accvgpr_mov_b32 a34, a0
; CHECK-NEXT: v_mov_b32_e32 v4, v3
; CHECK-NEXT: v_mov_b32_e32 v5, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[34:49], v[134:135], v[134:135], a[34:49]
; CHECK-NEXT: v_mov_b32_e32 v6, v3
; CHECK-NEXT: v_mov_b32_e32 v7, v3
; CHECK-NEXT: v_mov_b32_e32 v8, v3
@@ -54,56 +85,120 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: v_mov_b32_e32 v15, v3
; CHECK-NEXT: v_mov_b32_e32 v16, v3
; CHECK-NEXT: v_mov_b32_e32 v17, v3
-; CHECK-NEXT: v_mov_b32_e32 v214, v213
-; CHECK-NEXT: v_mov_b32_e32 v215, v213
-; CHECK-NEXT: v_mov_b32_e32 v216, v213
-; CHECK-NEXT: v_mov_b32_e32 v217, v213
-; CHECK-NEXT: v_mov_b32_e32 v218, v213
-; CHECK-NEXT: v_mov_b32_e32 v219, v213
-; CHECK-NEXT: v_mov_b32_e32 v220, v213
-; CHECK-NEXT: v_mov_b32_e32 v221, v213
-; CHECK-NEXT: v_mov_b32_e32 v222, v213
-; CHECK-NEXT: v_mov_b32_e32 v223, v213
-; CHECK-NEXT: v_mov_b32_e32 v224, v213
-; CHECK-NEXT: v_mov_b32_e32 v225, v213
-; CHECK-NEXT: v_mov_b32_e32 v226, v213
-; CHECK-NEXT: v_mov_b32_e32 v227, v213
-; CHECK-NEXT: v_accvgpr_write_b32 a47, v17
-; CHECK-NEXT: v_accvgpr_write_b32 a46, v16
-; CHECK-NEXT: v_accvgpr_write_b32 a45, v15
-; CHECK-NEXT: v_accvgpr_write_b32 a44, v14
-; CHECK-NEXT: v_accvgpr_write_b32 a43, v13
-; CHECK-NEXT: v_accvgpr_write_b32 a42, v12
-; CHECK-NEXT: v_accvgpr_write_b32 a41, v11
-; CHECK-NEXT: v_accvgpr_write_b32 a40, v10
-; CHECK-NEXT: v_accvgpr_write_b32 a39, v9
-; CHECK-NEXT: v_accvgpr_write_b32 a38, v8
-; CHECK-NEXT: v_accvgpr_write_b32 a37, v7
-; CHECK-NEXT: v_accvgpr_write_b32 a36, v6
-; CHECK-NEXT: v_accvgpr_write_b32 a35, v5
-; CHECK-NEXT: v_accvgpr_write_b32 a34, v4
-; CHECK-NEXT: v_accvgpr_write_b32 a33, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a32, v2
-; CHECK-NEXT: v_accvgpr_write_b32 a48, v212
-; CHECK-NEXT: v_accvgpr_write_b32 a49, v213
-; CHECK-NEXT: v_accvgpr_write_b32 a50, v214
-; CHECK-NEXT: v_accvgpr_write_b32 a51, v215
-; CHECK-NEXT: v_accvgpr_write_b32 a52, v216
-; CHECK-NEXT: v_accvgpr_write_b32 a53, v217
-; CHECK-NEXT: v_accvgpr_write_b32 a54, v218
-; CHECK-NEXT: v_accvgpr_write_b32 a55, v219
-; CHECK-NEXT: v_accvgpr_write_b32 a56, v220
-; CHECK-NEXT: v_accvgpr_write_b32 a57, v221
-; CHECK-NEXT: v_accvgpr_write_b32 a58, v222
-; CHECK-NEXT: v_accvgpr_write_b32 a59, v223
-; CHECK-NEXT: v_accvgpr_write_b32 a60, v224
-; CHECK-NEXT: v_accvgpr_write_b32 a61, v225
-; CHECK-NEXT: v_accvgpr_write_b32 a62, v226
-; CHECK-NEXT: v_accvgpr_write_b32 a63, v227
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[32:47], v[30:31], v[30:31], a[32:47]
-; CHECK-NEXT: v_mov_b32_e32 v13, v70
+; CHECK-NEXT: v_accvgpr_write_b32 a113, v17
+; CHECK-NEXT: v_accvgpr_write_b32 a112, v16
+; CHECK-NEXT: v_accvgpr_write_b32 a111, v15
+; CHECK-NEXT: v_accvgpr_write_b32 a110, v14
+; CHECK-NEXT: v_accvgpr_write_b32 a109, v13
+; CHECK-NEXT: v_accvgpr_write_b32 a108, v12
+; CHECK-NEXT: v_accvgpr_write_b32 a107, v11
+; CHECK-NEXT: v_accvgpr_mov_b32 a65, a49
+; CHECK-NEXT: v_accvgpr_mov_b32 a64, a48
+; CHECK-NEXT: v_accvgpr_mov_b32 a63, a47
+; CHECK-NEXT: v_accvgpr_mov_b32 a62, a46
+; CHECK-NEXT: v_accvgpr_mov_b32 a61, a45
+; CHECK-NEXT: v_accvgpr_mov_b32 a60, a44
+; CHECK-NEXT: v_accvgpr_mov_b32 a59, a43
+; CHECK-NEXT: v_accvgpr_mov_b32 a58, a42
+; CHECK-NEXT: v_accvgpr_mov_b32 a57, a41
+; CHECK-NEXT: v_accvgpr_mov_b32 a56, a40
+; CHECK-NEXT: v_accvgpr_mov_b32 a55, a39
+; CHECK-NEXT: v_accvgpr_mov_b32 a54, a38
+; CHECK-NEXT: v_accvgpr_mov_b32 a53, a37
+; CHECK-NEXT: v_accvgpr_mov_b32 a52, a36
+; CHECK-NEXT: v_accvgpr_mov_b32 a51, a35
+; CHECK-NEXT: v_accvgpr_mov_b32 a50, a34
+; CHECK-NEXT: v_accvgpr_write_b32 a34, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a35, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a36, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a37, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a38, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a39, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a40, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a41, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a42, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a43, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a44, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a45, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a46, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a47, v3
+; CHECK-NEXT: v_accvgpr_mov_b32 a49, a47
+; CHECK-NEXT: v_accvgpr_mov_b32 a48, a46
+; CHECK-NEXT: v_accvgpr_mov_b32 a47, a45
+; CHECK-NEXT: v_accvgpr_mov_b32 a46, a44
+; CHECK-NEXT: v_accvgpr_mov_b32 a45, a43
+; CHECK-NEXT: v_accvgpr_mov_b32 a44, a42
+; CHECK-NEXT: v_accvgpr_mov_b32 a43, a41
+; CHECK-NEXT: v_accvgpr_mov_b32 a42, a40
+; CHECK-NEXT: v_accvgpr_mov_b32 a41, a39
+; CHECK-NEXT: v_accvgpr_mov_b32 a40, a38
+; CHECK-NEXT: v_accvgpr_mov_b32 a39, a37
+; CHECK-NEXT: v_accvgpr_mov_b32 a38, a36
+; CHECK-NEXT: v_accvgpr_mov_b32 a37, a35
+; CHECK-NEXT: v_accvgpr_mov_b32 a36, a34
+; CHECK-NEXT: v_accvgpr_mov_b32 a35, a33
+; CHECK-NEXT: v_accvgpr_mov_b32 a34, a32
+; CHECK-NEXT: v_accvgpr_write_b32 a106, v10
+; CHECK-NEXT: v_accvgpr_write_b32 a105, v9
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[34:49], v[134:135], v[134:135], a[34:49]
+; CHECK-NEXT: v_accvgpr_write_b32 a104, v8
+; CHECK-NEXT: v_accvgpr_write_b32 a103, v7
+; CHECK-NEXT: v_accvgpr_write_b32 a102, v6
+; CHECK-NEXT: v_accvgpr_write_b32 a101, v5
+; CHECK-NEXT: v_accvgpr_write_b32 a100, v4
+; CHECK-NEXT: v_accvgpr_write_b32 a99, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a98, v2
+; CHECK-NEXT: v_accvgpr_write_b32 a16, v24
+; CHECK-NEXT: v_accvgpr_write_b32 a18, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a19, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a20, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a21, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a22, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a23, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a24, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[98:113], v[134:135], v[134:135], a[98:113]
+; CHECK-NEXT: v_accvgpr_write_b32 a25, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a26, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a27, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a28, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a29, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a30, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a31, v3
+; CHECK-NEXT: v_accvgpr_read_b32 v24, a34
+; CHECK-NEXT: v_accvgpr_read_b32 v25, a35
+; CHECK-NEXT: v_accvgpr_read_b32 v26, a36
+; CHECK-NEXT: v_accvgpr_read_b32 v27, a37
+; CHECK-NEXT: v_accvgpr_read_b32 v28, a38
+; CHECK-NEXT: v_accvgpr_read_b32 v29, a39
+; CHECK-NEXT: v_accvgpr_read_b32 v30, a40
+; CHECK-NEXT: v_accvgpr_read_b32 v31, a41
+; CHECK-NEXT: v_accvgpr_read_b32 v32, a42
+; CHECK-NEXT: v_accvgpr_read_b32 v33, a43
+; CHECK-NEXT: v_accvgpr_read_b32 v34, a44
+; CHECK-NEXT: v_accvgpr_read_b32 v35, a45
+; CHECK-NEXT: v_accvgpr_read_b32 v36, a46
+; CHECK-NEXT: v_accvgpr_read_b32 v37, a47
+; CHECK-NEXT: v_accvgpr_read_b32 v38, a48
+; CHECK-NEXT: v_accvgpr_read_b32 v39, a49
+; CHECK-NEXT: v_accvgpr_mov_b32 a49, a31
+; CHECK-NEXT: v_accvgpr_mov_b32 a48, a30
+; CHECK-NEXT: v_accvgpr_mov_b32 a47, a29
+; CHECK-NEXT: v_accvgpr_mov_b32 a46, a28
+; CHECK-NEXT: v_accvgpr_mov_b32 a45, a27
+; CHECK-NEXT: v_accvgpr_mov_b32 a44, a26
+; CHECK-NEXT: v_accvgpr_mov_b32 a43, a25
+; CHECK-NEXT: v_accvgpr_mov_b32 a42, a24
+; CHECK-NEXT: v_accvgpr_mov_b32 a41, a23
+; CHECK-NEXT: v_accvgpr_mov_b32 a40, a22
+; CHECK-NEXT: v_accvgpr_mov_b32 a39, a21
+; CHECK-NEXT: v_accvgpr_mov_b32 a38, a20
+; CHECK-NEXT: v_accvgpr_mov_b32 a37, a19
+; CHECK-NEXT: v_accvgpr_mov_b32 a36, a18
+; CHECK-NEXT: v_accvgpr_mov_b32 a35, a17
+; CHECK-NEXT: v_accvgpr_mov_b32 a34, a16
; CHECK-NEXT: v_mov_b32_e32 v68, v3
; CHECK-NEXT: v_mov_b32_e32 v69, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[34:49], v[134:135], v[134:135], a[34:49]
; CHECK-NEXT: v_mov_b32_e32 v70, v3
; CHECK-NEXT: v_mov_b32_e32 v71, v3
; CHECK-NEXT: v_mov_b32_e32 v72, v3
@@ -116,11 +211,98 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: v_mov_b32_e32 v79, v3
; CHECK-NEXT: v_mov_b32_e32 v80, v3
; CHECK-NEXT: v_mov_b32_e32 v81, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[48:63], v[30:31], v[30:31], a[48:63]
-; CHECK-NEXT: v_mov_b32_e32 v242, v32
; CHECK-NEXT: v_mov_b32_e32 v1, v3
; CHECK-NEXT: v_mov_b32_e32 v2, v3
-; CHECK-NEXT: v_mov_b32_e32 v14, v20
+; CHECK-NEXT: v_mov_b32_e32 v13, v22
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[66:81], v[134:135], v[134:135], v[66:81]
+; CHECK-NEXT: v_mov_b32_e32 v14, v18
+; CHECK-NEXT: v_mov_b32_e32 v115, v3
+; CHECK-NEXT: v_mov_b32_e32 v116, v3
+; CHECK-NEXT: v_mov_b32_e32 v117, v3
+; CHECK-NEXT: v_mov_b32_e32 v118, v3
+; CHECK-NEXT: v_mov_b32_e32 v119, v3
+; CHECK-NEXT: v_mov_b32_e32 v120, v3
+; CHECK-NEXT: v_mov_b32_e32 v121, v3
+; CHECK-NEXT: v_mov_b32_e32 v122, v3
+; CHECK-NEXT: v_mov_b32_e32 v123, v3
+; CHECK-NEXT: v_mov_b32_e32 v124, v3
+; CHECK-NEXT: v_mov_b32_e32 v125, v3
+; CHECK-NEXT: v_mov_b32_e32 v126, v3
+; CHECK-NEXT: v_mov_b32_e32 v127, v3
+; CHECK-NEXT: v_mov_b32_e32 v128, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[98:113], v[134:135], v[134:135], a[98:113]
+; CHECK-NEXT: v_mov_b32_e32 v129, v3
+; CHECK-NEXT: v_accvgpr_mov_b32 a81, a49
+; CHECK-NEXT: v_accvgpr_mov_b32 a80, a48
+; CHECK-NEXT: v_accvgpr_mov_b32 a79, a47
+; CHECK-NEXT: v_accvgpr_mov_b32 a78, a46
+; CHECK-NEXT: v_accvgpr_mov_b32 a77, a45
+; CHECK-NEXT: v_accvgpr_mov_b32 a76, a44
+; CHECK-NEXT: v_accvgpr_mov_b32 a75, a43
+; CHECK-NEXT: v_accvgpr_mov_b32 a74, a42
+; CHECK-NEXT: v_accvgpr_mov_b32 a73, a41
+; CHECK-NEXT: v_accvgpr_mov_b32 a72, a40
+; CHECK-NEXT: v_accvgpr_mov_b32 a71, a39
+; CHECK-NEXT: v_accvgpr_mov_b32 a70, a38
+; CHECK-NEXT: v_accvgpr_mov_b32 a69, a37
+; CHECK-NEXT: v_accvgpr_mov_b32 a68, a36
+; CHECK-NEXT: v_accvgpr_mov_b32 a67, a35
+; CHECK-NEXT: v_accvgpr_mov_b32 a66, a34
+; CHECK-NEXT: v_accvgpr_write_b32 a97, v15
+; CHECK-NEXT: v_accvgpr_write_b32 a96, v14
+; CHECK-NEXT: v_accvgpr_write_b32 a95, v13
+; CHECK-NEXT: v_accvgpr_write_b32 a94, v12
+; CHECK-NEXT: v_accvgpr_write_b32 a93, v11
+; CHECK-NEXT: v_accvgpr_write_b32 a92, v10
+; CHECK-NEXT: v_accvgpr_write_b32 a91, v9
+; CHECK-NEXT: v_accvgpr_write_b32 a90, v8
+; CHECK-NEXT: v_accvgpr_write_b32 a89, v7
+; CHECK-NEXT: v_accvgpr_write_b32 a88, v6
+; CHECK-NEXT: v_accvgpr_write_b32 a87, v5
+; CHECK-NEXT: v_accvgpr_write_b32 a86, v4
+; CHECK-NEXT: v_accvgpr_write_b32 a85, v3
+; CHECK-NEXT: v_accvgpr_write_b32 a84, v2
+; CHECK-NEXT: v_accvgpr_write_b32 a83, v1
+; CHECK-NEXT: v_accvgpr_write_b32 a82, v0
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[114:129], v[134:135], v[134:135], v[114:129]
+; CHECK-NEXT: v_accvgpr_mov_b32 a34, a98
+; CHECK-NEXT: v_accvgpr_mov_b32 a35, a99
+; CHECK-NEXT: v_accvgpr_mov_b32 a36, a100
+; CHECK-NEXT: v_accvgpr_mov_b32 a37, a101
+; CHECK-NEXT: v_accvgpr_mov_b32 a38, a102
+; CHECK-NEXT: v_accvgpr_mov_b32 a39, a103
+; CHECK-NEXT: v_accvgpr_mov_b32 a40, a104
+; CHECK-NEXT: v_accvgpr_mov_b32 a41, a105
+; CHECK-NEXT: v_accvgpr_mov_b32 a42, a106
+; CHECK-NEXT: v_accvgpr_mov_b32 a43, a107
+; CHECK-NEXT: v_accvgpr_mov_b32 a44, a108
+; CHECK-NEXT: v_accvgpr_mov_b32 a45, a109
+; CHECK-NEXT: v_accvgpr_mov_b32 a46, a110
+; CHECK-NEXT: v_accvgpr_mov_b32 a47, a111
+; CHECK-NEXT: v_accvgpr_mov_b32 a48, a112
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[82:97], v[134:135], v[134:135], a[82:97]
+; CHECK-NEXT: v_accvgpr_mov_b32 a49, a113
+; CHECK-NEXT: v_accvgpr_write_b32 a113, v39
+; CHECK-NEXT: v_accvgpr_write_b32 a112, v38
+; CHECK-NEXT: v_accvgpr_write_b32 a111, v37
+; CHECK-NEXT: v_accvgpr_write_b32 a110, v36
+; CHECK-NEXT: v_accvgpr_write_b32 a109, v35
+; CHECK-NEXT: v_accvgpr_write_b32 a108, v34
+; CHECK-NEXT: v_accvgpr_write_b32 a107, v33
+; CHECK-NEXT: v_accvgpr_write_b32 a106, v32
+; CHECK-NEXT: v_accvgpr_write_b32 a105, v31
+; CHECK-NEXT: v_accvgpr_write_b32 a104, v30
+; CHECK-NEXT: v_accvgpr_write_b32 a103, v29
+; CHECK-NEXT: v_accvgpr_write_b32 a102, v28
+; CHECK-NEXT: v_accvgpr_write_b32 a101, v27
+; CHECK-NEXT: v_accvgpr_write_b32 a100, v26
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[66:81], v[134:135], v[134:135], a[66:81]
+; CHECK-NEXT: v_accvgpr_write_b32 a99, v25
+; CHECK-NEXT: v_accvgpr_write_b32 a98, v24
+; CHECK-NEXT: v_mov_b32_e32 v5, v20
+; CHECK-NEXT: v_mov_b32_e32 v13, v3
+; CHECK-NEXT: v_mov_b32_e32 v14, v3
+; CHECK-NEXT: v_mov_b32_e32 v242, v226
; CHECK-NEXT: v_mov_b32_e32 v228, v3
; CHECK-NEXT: v_mov_b32_e32 v229, v3
; CHECK-NEXT: v_mov_b32_e32 v230, v3
@@ -130,183 +312,23 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: v_mov_b32_e32 v234, v3
; CHECK-NEXT: v_mov_b32_e32 v235, v3
; CHECK-NEXT: v_mov_b32_e32 v236, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[116:131], v[134:135], v[134:135], v[66:81]
; CHECK-NEXT: v_mov_b32_e32 v237, v3
; CHECK-NEXT: v_mov_b32_e32 v238, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[66:81], v[30:31], v[30:31], v[66:81]
; CHECK-NEXT: v_mov_b32_e32 v239, v3
; CHECK-NEXT: v_mov_b32_e32 v240, v3
; CHECK-NEXT: v_mov_b32_e32 v241, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a18, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a19, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a20, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a21, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a22, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a23, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a24, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a25, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a26, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a27, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a28, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a29, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a30, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a31, v3
-; CHECK-NEXT: v_accvgpr_mov_b32 a0, a64
-; CHECK-NEXT: v_accvgpr_write_b32 a95, v15
-; CHECK-NEXT: v_accvgpr_write_b32 a94, v14
-; CHECK-NEXT: v_accvgpr_write_b32 a93, v13
-; CHECK-NEXT: v_accvgpr_write_b32 a92, v12
-; CHECK-NEXT: v_accvgpr_write_b32 a91, v11
-; CHECK-NEXT: v_accvgpr_write_b32 a90, v10
-; CHECK-NEXT: v_accvgpr_write_b32 a89, v9
-; CHECK-NEXT: v_accvgpr_write_b32 a88, v8
-; CHECK-NEXT: v_accvgpr_write_b32 a87, v7
-; CHECK-NEXT: v_accvgpr_write_b32 a86, v6
-; CHECK-NEXT: v_accvgpr_write_b32 a85, v5
-; CHECK-NEXT: v_accvgpr_write_b32 a84, v4
-; CHECK-NEXT: v_accvgpr_write_b32 a83, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a82, v2
-; CHECK-NEXT: v_accvgpr_write_b32 a81, v1
-; CHECK-NEXT: v_accvgpr_write_b32 a80, v0
-; CHECK-NEXT: v_mov_b32_e32 v5, v86
-; CHECK-NEXT: v_mov_b32_e32 v13, v3
-; CHECK-NEXT: v_mov_b32_e32 v14, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a64, v228
-; CHECK-NEXT: v_accvgpr_write_b32 a65, v229
-; CHECK-NEXT: v_accvgpr_write_b32 a66, v230
-; CHECK-NEXT: v_accvgpr_write_b32 a67, v231
-; CHECK-NEXT: v_accvgpr_write_b32 a68, v232
-; CHECK-NEXT: v_accvgpr_write_b32 a69, v233
-; CHECK-NEXT: v_accvgpr_write_b32 a70, v234
-; CHECK-NEXT: v_accvgpr_write_b32 a71, v235
-; CHECK-NEXT: v_accvgpr_write_b32 a72, v236
-; CHECK-NEXT: v_accvgpr_write_b32 a73, v237
-; CHECK-NEXT: v_accvgpr_write_b32 a74, v238
-; CHECK-NEXT: v_accvgpr_write_b32 a75, v239
-; CHECK-NEXT: v_accvgpr_write_b32 a76, v240
-; CHECK-NEXT: v_accvgpr_write_b32 a77, v241
-; CHECK-NEXT: v_accvgpr_write_b32 a78, v242
-; CHECK-NEXT: v_accvgpr_write_b32 a79, v243
-; CHECK-NEXT: v_accvgpr_mov_b32 a111, a47
-; CHECK-NEXT: v_accvgpr_mov_b32 a110, a46
-; CHECK-NEXT: v_accvgpr_mov_b32 a109, a45
-; CHECK-NEXT: v_accvgpr_mov_b32 a108, a44
-; CHECK-NEXT: v_accvgpr_mov_b32 a107, a43
-; CHECK-NEXT: v_accvgpr_mov_b32 a106, a42
-; CHECK-NEXT: v_accvgpr_mov_b32 a105, a41
-; CHECK-NEXT: v_accvgpr_mov_b32 a104, a40
-; CHECK-NEXT: v_accvgpr_mov_b32 a103, a39
-; CHECK-NEXT: v_accvgpr_mov_b32 a102, a38
-; CHECK-NEXT: v_accvgpr_mov_b32 a101, a37
-; CHECK-NEXT: v_accvgpr_mov_b32 a100, a36
-; CHECK-NEXT: v_accvgpr_mov_b32 a99, a35
-; CHECK-NEXT: v_accvgpr_mov_b32 a98, a34
-; CHECK-NEXT: v_accvgpr_mov_b32 a97, a33
-; CHECK-NEXT: v_accvgpr_mov_b32 a96, a32
-; CHECK-NEXT: v_accvgpr_mov_b32 a32, a48
-; CHECK-NEXT: v_accvgpr_mov_b32 a33, a49
-; CHECK-NEXT: v_accvgpr_mov_b32 a34, a50
-; CHECK-NEXT: v_accvgpr_mov_b32 a35, a51
-; CHECK-NEXT: v_accvgpr_mov_b32 a36, a52
-; CHECK-NEXT: v_accvgpr_mov_b32 a37, a53
-; CHECK-NEXT: v_accvgpr_mov_b32 a38, a54
-; CHECK-NEXT: v_accvgpr_mov_b32 a39, a55
-; CHECK-NEXT: v_accvgpr_mov_b32 a40, a56
-; CHECK-NEXT: v_accvgpr_mov_b32 a41, a57
-; CHECK-NEXT: v_accvgpr_mov_b32 a42, a58
-; CHECK-NEXT: v_accvgpr_mov_b32 a43, a59
-; CHECK-NEXT: v_accvgpr_mov_b32 a44, a60
-; CHECK-NEXT: v_accvgpr_mov_b32 a45, a61
-; CHECK-NEXT: v_accvgpr_mov_b32 a46, a62
-; CHECK-NEXT: v_accvgpr_mov_b32 a47, a63
-; CHECK-NEXT: v_accvgpr_mov_b32 a63, a31
-; CHECK-NEXT: v_accvgpr_mov_b32 a62, a30
-; CHECK-NEXT: v_accvgpr_mov_b32 a61, a29
-; CHECK-NEXT: v_accvgpr_mov_b32 a60, a28
-; CHECK-NEXT: v_accvgpr_mov_b32 a59, a27
-; CHECK-NEXT: v_accvgpr_mov_b32 a58, a26
-; CHECK-NEXT: v_accvgpr_mov_b32 a57, a25
-; CHECK-NEXT: v_accvgpr_mov_b32 a56, a24
-; CHECK-NEXT: v_accvgpr_mov_b32 a55, a23
-; CHECK-NEXT: v_accvgpr_mov_b32 a54, a22
-; CHECK-NEXT: v_accvgpr_mov_b32 a53, a21
-; CHECK-NEXT: v_accvgpr_mov_b32 a52, a20
-; CHECK-NEXT: v_accvgpr_mov_b32 a51, a19
-; CHECK-NEXT: v_accvgpr_mov_b32 a50, a18
-; CHECK-NEXT: v_accvgpr_mov_b32 a49, a17
-; CHECK-NEXT: v_accvgpr_mov_b32 a48, a16
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[64:79], v[30:31], v[30:31], a[64:79]
-; CHECK-NEXT: v_pk_mov_b32 v[100:101], v[16:17], v[16:17] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[98:99], v[14:15], v[14:15] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[96:97], v[12:13], v[12:13] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[94:95], v[10:11], v[10:11] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[92:93], v[8:9], v[8:9] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[90:91], v[6:7], v[6:7] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[88:89], v[4:5], v[4:5] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[86:87], v[2:3], v[2:3] op_sel:[0,1]
-; CHECK-NEXT: v_mov_b32_e32 v36, v3
-; CHECK-NEXT: v_mov_b32_e32 v37, v3
-; CHECK-NEXT: v_mov_b32_e32 v38, v3
-; CHECK-NEXT: v_mov_b32_e32 v39, v3
-; CHECK-NEXT: v_mov_b32_e32 v40, v3
-; CHECK-NEXT: v_mov_b32_e32 v41, v3
-; CHECK-NEXT: v_mov_b32_e32 v42, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[48:63], v[30:31], v[30:31], a[48:63]
-; CHECK-NEXT: v_mov_b32_e32 v43, v3
-; CHECK-NEXT: v_mov_b32_e32 v44, v3
-; CHECK-NEXT: v_mov_b32_e32 v45, v3
-; CHECK-NEXT: v_mov_b32_e32 v46, v3
-; CHECK-NEXT: v_mov_b32_e32 v47, v3
-; CHECK-NEXT: v_mov_b32_e32 v48, v3
-; CHECK-NEXT: v_mov_b32_e32 v49, v3
-; CHECK-NEXT: v_pk_mov_b32 v[228:229], v[48:49], v[48:49] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[226:227], v[46:47], v[46:47] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[224:225], v[44:45], v[44:45] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[222:223], v[42:43], v[42:43] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[220:221], v[40:41], v[40:41] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[218:219], v[38:39], v[38:39] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[216:217], v[36:37], v[36:37] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[214:215], v[34:35], v[34:35] op_sel:[0,1]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[30:31], v[30:31], v[66:81]
-; CHECK-NEXT: v_pk_mov_b32 v[32:33], v[214:215], v[214:215] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[34:35], v[216:217], v[216:217] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[36:37], v[218:219], v[218:219] op_sel:[0,1]
-; CHECK-NEXT: v_accvgpr_mov_b32 a127, a63
-; CHECK-NEXT: v_accvgpr_mov_b32 a126, a62
-; CHECK-NEXT: v_accvgpr_mov_b32 a125, a61
-; CHECK-NEXT: v_accvgpr_mov_b32 a124, a60
-; CHECK-NEXT: v_accvgpr_mov_b32 a123, a59
-; CHECK-NEXT: v_accvgpr_mov_b32 a122, a58
-; CHECK-NEXT: v_accvgpr_mov_b32 a121, a57
-; CHECK-NEXT: v_accvgpr_mov_b32 a120, a56
-; CHECK-NEXT: v_accvgpr_mov_b32 a119, a55
-; CHECK-NEXT: v_accvgpr_mov_b32 a118, a54
-; CHECK-NEXT: v_accvgpr_mov_b32 a117, a53
-; CHECK-NEXT: v_accvgpr_mov_b32 a116, a52
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[80:95], v[30:31], v[30:31], a[80:95]
-; CHECK-NEXT: v_accvgpr_mov_b32 a115, a51
-; CHECK-NEXT: v_accvgpr_mov_b32 a114, a50
-; CHECK-NEXT: v_accvgpr_mov_b32 a113, a49
-; CHECK-NEXT: v_accvgpr_mov_b32 a112, a48
-; CHECK-NEXT: v_accvgpr_mov_b32 a48, a64
-; CHECK-NEXT: v_accvgpr_mov_b32 a49, a65
-; CHECK-NEXT: v_accvgpr_mov_b32 a50, a66
-; CHECK-NEXT: v_accvgpr_mov_b32 a51, a67
-; CHECK-NEXT: v_accvgpr_mov_b32 a52, a68
-; CHECK-NEXT: v_accvgpr_mov_b32 a53, a69
-; CHECK-NEXT: v_accvgpr_mov_b32 a54, a70
-; CHECK-NEXT: v_accvgpr_mov_b32 a55, a71
-; CHECK-NEXT: v_accvgpr_mov_b32 a56, a72
-; CHECK-NEXT: v_accvgpr_mov_b32 a57, a73
-; CHECK-NEXT: v_accvgpr_mov_b32 a58, a74
-; CHECK-NEXT: v_accvgpr_mov_b32 a59, a75
-; CHECK-NEXT: v_accvgpr_mov_b32 a60, a76
-; CHECK-NEXT: v_accvgpr_mov_b32 a61, a77
-; CHECK-NEXT: v_accvgpr_mov_b32 a62, a78
-; CHECK-NEXT: v_accvgpr_mov_b32 a63, a79
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[30:31], v[30:31], v[4:19]
-; CHECK-NEXT: v_accvgpr_mov_b32 a64, a80
-; CHECK-NEXT: v_accvgpr_mov_b32 a65, a81
+; CHECK-NEXT: v_pk_mov_b32 v[18:19], v[16:17], v[16:17] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[16:17], v[14:15], v[14:15] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[14:15], v[12:13], v[12:13] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[12:13], v[10:11], v[10:11] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[10:11], v[8:9], v[8:9] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[8:9], v[6:7], v[6:7] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[6:7], v[4:5], v[4:5] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[4:5], v[2:3], v[2:3] op_sel:[0,1]
+; CHECK-NEXT: v_accvgpr_read_b32 v24, a66
; CHECK-NEXT: v_accvgpr_mov_b32 a66, a82
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[50:65], v[134:135], v[134:135], a[50:65]
; CHECK-NEXT: v_accvgpr_mov_b32 a67, a83
; CHECK-NEXT: v_accvgpr_mov_b32 a68, a84
; CHECK-NEXT: v_accvgpr_mov_b32 a69, a85
@@ -319,45 +341,18 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: v_accvgpr_mov_b32 a76, a92
; CHECK-NEXT: v_accvgpr_mov_b32 a77, a93
; CHECK-NEXT: v_accvgpr_mov_b32 a78, a94
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[48:63], v[30:31], v[30:31], a[48:63]
; CHECK-NEXT: v_accvgpr_mov_b32 a79, a95
-; CHECK-NEXT: v_pk_mov_b32 v[38:39], v[220:221], v[220:221] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[40:41], v[222:223], v[222:223] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[42:43], v[224:225], v[224:225] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[44:45], v[226:227], v[226:227] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[46:47], v[228:229], v[228:229] op_sel:[0,1]
-; CHECK-NEXT: v_mov_b32_e32 v130, v3
-; CHECK-NEXT: v_mov_b32_e32 v131, v3
-; CHECK-NEXT: v_mov_b32_e32 v132, v3
-; CHECK-NEXT: v_mov_b32_e32 v133, v3
-; CHECK-NEXT: v_mov_b32_e32 v134, v3
-; CHECK-NEXT: v_mov_b32_e32 v135, v3
-; CHECK-NEXT: v_mov_b32_e32 v136, v3
-; CHECK-NEXT: v_mov_b32_e32 v137, v3
-; CHECK-NEXT: v_mov_b32_e32 v138, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[64:79], v[30:31], v[30:31], a[64:79]
-; CHECK-NEXT: v_mov_b32_e32 v139, v3
-; CHECK-NEXT: v_mov_b32_e32 v140, v3
-; CHECK-NEXT: v_mov_b32_e32 v141, v3
-; CHECK-NEXT: v_mov_b32_e32 v142, v3
-; CHECK-NEXT: v_mov_b32_e32 v143, v3
-; CHECK-NEXT: v_mov_b32_e32 v144, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a2, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a3, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a4, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a5, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a6, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a7, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a8, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a9, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a10, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[86:101], v[30:31], v[30:31], v[86:101]
-; CHECK-NEXT: v_accvgpr_write_b32 a11, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a12, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a13, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a14, v3
-; CHECK-NEXT: v_accvgpr_write_b32 a15, v3
-; CHECK-NEXT: v_mov_b32_e32 v163, v3
+; CHECK-NEXT: v_accvgpr_mov_b32 a80, a96
+; CHECK-NEXT: v_accvgpr_mov_b32 a81, a97
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[34:49], v[134:135], v[134:135], a[34:49]
+; CHECK-NEXT: v_pk_mov_b32 v[226:227], v[228:229], v[228:229] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[228:229], v[230:231], v[230:231] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[230:231], v[232:233], v[232:233] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[232:233], v[234:235], v[234:235] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[234:235], v[236:237], v[236:237] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[236:237], v[238:239], v[238:239] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[238:239], v[240:241], v[240:241] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[240:241], v[242:243], v[242:243] op_sel:[0,1]
; CHECK-NEXT: v_mov_b32_e32 v164, v3
; CHECK-NEXT: v_mov_b32_e32 v165, v3
; CHECK-NEXT: v_mov_b32_e32 v166, v3
@@ -365,17 +360,39 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: v_mov_b32_e32 v168, v3
; CHECK-NEXT: v_mov_b32_e32 v169, v3
; CHECK-NEXT: v_mov_b32_e32 v170, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[98:113], v[134:135], v[134:135], a[98:113]
; CHECK-NEXT: v_mov_b32_e32 v171, v3
; CHECK-NEXT: v_mov_b32_e32 v172, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[32:47], v[30:31], v[30:31], v[32:47]
; CHECK-NEXT: v_mov_b32_e32 v173, v3
+; CHECK-NEXT: v_accvgpr_read_b32 v42, a34
+; CHECK-NEXT: v_accvgpr_mov_b32 a34, a50
+; CHECK-NEXT: v_accvgpr_mov_b32 a35, a51
+; CHECK-NEXT: v_accvgpr_mov_b32 a36, a52
+; CHECK-NEXT: v_accvgpr_mov_b32 a37, a53
+; CHECK-NEXT: v_accvgpr_mov_b32 a38, a54
+; CHECK-NEXT: v_accvgpr_mov_b32 a39, a55
+; CHECK-NEXT: v_accvgpr_mov_b32 a40, a56
+; CHECK-NEXT: v_accvgpr_mov_b32 a41, a57
+; CHECK-NEXT: v_accvgpr_mov_b32 a42, a58
+; CHECK-NEXT: v_accvgpr_mov_b32 a43, a59
+; CHECK-NEXT: v_accvgpr_mov_b32 a44, a60
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[134:135], v[134:135], v[4:19]
+; CHECK-NEXT: v_accvgpr_mov_b32 a45, a61
+; CHECK-NEXT: v_accvgpr_mov_b32 a46, a62
+; CHECK-NEXT: v_accvgpr_mov_b32 a47, a63
+; CHECK-NEXT: v_accvgpr_mov_b32 a48, a64
+; CHECK-NEXT: v_accvgpr_mov_b32 a49, a65
; CHECK-NEXT: v_mov_b32_e32 v174, v3
; CHECK-NEXT: v_mov_b32_e32 v175, v3
; CHECK-NEXT: v_mov_b32_e32 v176, v3
; CHECK-NEXT: v_mov_b32_e32 v177, v3
+; CHECK-NEXT: v_accvgpr_mov_b32 a50, a112
+; CHECK-NEXT: v_accvgpr_mov_b32 a51, a113
+; CHECK-NEXT: v_mov_b32_e32 v146, v3
; CHECK-NEXT: v_mov_b32_e32 v147, v3
; CHECK-NEXT: v_mov_b32_e32 v148, v3
; CHECK-NEXT: v_mov_b32_e32 v149, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[66:81], v[134:135], v[134:135], a[66:81]
; CHECK-NEXT: v_mov_b32_e32 v150, v3
; CHECK-NEXT: v_mov_b32_e32 v151, v3
; CHECK-NEXT: v_mov_b32_e32 v152, v3
@@ -383,177 +400,136 @@ define amdgpu_kernel void @multi_store_spill_slot() #0 {
; CHECK-NEXT: v_mov_b32_e32 v154, v3
; CHECK-NEXT: v_mov_b32_e32 v155, v3
; CHECK-NEXT: v_mov_b32_e32 v156, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[30:31], v[30:31], v[4:19]
; CHECK-NEXT: v_mov_b32_e32 v157, v3
; CHECK-NEXT: v_mov_b32_e32 v158, v3
; CHECK-NEXT: v_mov_b32_e32 v159, v3
; CHECK-NEXT: v_mov_b32_e32 v160, v3
-; CHECK-NEXT: v_mov_b32_e32 v161, v3
-; CHECK-NEXT: v_accvgpr_read_b32 v68, a64
+; CHECK-NEXT: v_mov_b32_e32 v99, v3
+; CHECK-NEXT: v_mov_b32_e32 v100, v3
+; CHECK-NEXT: v_mov_b32_e32 v101, v3
+; CHECK-NEXT: v_mov_b32_e32 v102, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[116:131], v[134:135], v[134:135], v[116:131]
+; CHECK-NEXT: v_mov_b32_e32 v103, v3
+; CHECK-NEXT: v_mov_b32_e32 v104, v3
+; CHECK-NEXT: v_mov_b32_e32 v105, v3
+; CHECK-NEXT: v_mov_b32_e32 v106, v3
+; CHECK-NEXT: v_mov_b32_e32 v107, v3
+; CHECK-NEXT: v_mov_b32_e32 v108, v3
+; CHECK-NEXT: v_mov_b32_e32 v109, v3
+; CHECK-NEXT: v_mov_b32_e32 v110, v3
+; CHECK-NEXT: v_mov_b32_e32 v111, v3
+; CHECK-NEXT: v_mov_b32_e32 v112, v3
+; CHECK-NEXT: v_mov_b32_e32 v113, v3
+; CHECK-NEXT: v_mov_b32_e32 v181, v3
+; CHECK-NEXT: v_mov_b32_e32 v182, v3
+; CHECK-NEXT: v_mov_b32_e32 v183, v3
+; CHECK-NEXT: v_mov_b32_e32 v184, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[226:241], v[134:135], v[134:135], v[226:241]
+; CHECK-NEXT: v_mov_b32_e32 v185, v3
+; CHECK-NEXT: v_mov_b32_e32 v186, v3
+; CHECK-NEXT: v_mov_b32_e32 v187, v3
+; CHECK-NEXT: v_mov_b32_e32 v188, v3
+; CHECK-NEXT: v_mov_b32_e32 v189, v3
+; CHECK-NEXT: v_mov_b32_e32 v190, v3
+; CHECK-NEXT: v_mov_b32_e32 v191, v3
+; CHECK-NEXT: v_mov_b32_e32 v192, v3
+; CHECK-NEXT: v_mov_b32_e32 v193, v3
+; CHECK-NEXT: v_mov_b32_e32 v194, v3
+; CHECK-NEXT: v_mov_b32_e32 v195, v3
; CHECK-NEXT: v_mov_b32_e32 v51, v3
; CHECK-NEXT: v_mov_b32_e32 v52, v3
; CHECK-NEXT: v_mov_b32_e32 v53, v3
; CHECK-NEXT: v_mov_b32_e32 v54, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[34:49], v[134:135], v[134:135], a[34:49]
; CHECK-NEXT: v_mov_b32_e32 v55, v3
; CHECK-NEXT: v_mov_b32_e32 v56, v3
; CHECK-NEXT: v_mov_b32_e32 v57, v3
; CHECK-NEXT: v_mov_b32_e32 v58, v3
; CHECK-NEXT: v_mov_b32_e32 v59, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[32:47], v[30:31], v[30:31], a[32:47]
; CHECK-NEXT: v_mov_b32_e32 v60, v3
; CHECK-NEXT: v_mov_b32_e32 v61, v3
; CHECK-NEXT: v_mov_b32_e32 v62, v3
; CHECK-NEXT: v_mov_b32_e32 v63, v3
; CHECK-NEXT: v_mov_b32_e32 v64, v3
; CHECK-NEXT: v_mov_b32_e32 v65, v3
-; CHECK-NEXT: v_mov_b32_e32 v197, v3
-; CHECK-NEXT: v_mov_b32_e32 v198, v3
-; CHECK-NEXT: v_mov_b32_e32 v199, v3
-; CHECK-NEXT: v_mov_b32_e32 v200, v3
-; CHECK-NEXT: v_mov_b32_e32 v201, v3
-; CHECK-NEXT: v_mov_b32_e32 v202, v3
-; CHECK-NEXT: v_mov_b32_e32 v203, v3
-; CHECK-NEXT: v_mov_b32_e32 v204, v3
-; CHECK-NEXT: v_mov_b32_e32 v205, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[48:63], v[30:31], v[30:31], a[48:63]
-; CHECK-NEXT: v_mov_b32_e32 v206, v3
-; CHECK-NEXT: v_mov_b32_e32 v207, v3
-; CHECK-NEXT: v_mov_b32_e32 v208, v3
-; CHECK-NEXT: v_mov_b32_e32 v209, v3
-; CHECK-NEXT: v_mov_b32_e32 v210, v3
-; CHECK-NEXT: v_mov_b32_e32 v211, v3
-; CHECK-NEXT: v_pk_mov_b32 v[228:229], v[46:47], v[46:47] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[226:227], v[44:45], v[44:45] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[224:225], v[42:43], v[42:43] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[222:223], v[40:41], v[40:41] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[220:221], v[38:39], v[38:39] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[218:219], v[36:37], v[36:37] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[216:217], v[34:35], v[34:35] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[214:215], v[32:33], v[32:33] op_sel:[0,1]
-; CHECK-NEXT: v_accvgpr_mov_b32 a18, a32
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[112:127], v[30:31], v[30:31], a[112:127]
-; CHECK-NEXT: v_accvgpr_mov_b32 a19, a33
-; CHECK-NEXT: v_accvgpr_mov_b32 a20, a34
-; CHECK-NEXT: v_accvgpr_mov_b32 a21, a35
-; CHECK-NEXT: v_accvgpr_read_b32 v32, a48
-; CHECK-NEXT: v_accvgpr_mov_b32 a22, a36
-; CHECK-NEXT: v_accvgpr_mov_b32 a23, a37
-; CHECK-NEXT: v_accvgpr_mov_b32 a24, a38
-; CHECK-NEXT: v_accvgpr_mov_b32 a25, a39
-; CHECK-NEXT: v_accvgpr_mov_b32 a26, a40
-; CHECK-NEXT: v_accvgpr_mov_b32 a27, a41
-; CHECK-NEXT: v_accvgpr_mov_b32 a28, a42
-; CHECK-NEXT: v_accvgpr_mov_b32 a29, a43
-; CHECK-NEXT: v_accvgpr_mov_b32 a30, a44
-; CHECK-NEXT: v_accvgpr_mov_b32 a31, a45
-; CHECK-NEXT: v_accvgpr_mov_b32 a32, a46
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[130:145], v[30:31], v[30:31], v[130:145]
-; CHECK-NEXT: v_accvgpr_mov_b32 a33, a47
-; CHECK-NEXT: v_pk_mov_b32 v[20:21], v[222:223], v[222:223] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[22:23], v[224:225], v[224:225] op_sel:[0,1]
-; CHECK-NEXT: v_accvgpr_mov_b32 a48, a112
-; CHECK-NEXT: v_accvgpr_mov_b32 a49, a113
-; CHECK-NEXT: v_accvgpr_mov_b32 a50, a114
-; CHECK-NEXT: v_accvgpr_mov_b32 a51, a115
-; CHECK-NEXT: v_accvgpr_mov_b32 a52, a116
-; CHECK-NEXT: v_accvgpr_mov_b32 a53, a117
-; CHECK-NEXT: v_accvgpr_mov_b32 a54, a118
-; CHECK-NEXT: v_accvgpr_mov_b32 a55, a119
-; CHECK-NEXT: v_accvgpr_mov_b32 a56, a120
-; CHECK-NEXT: v_accvgpr_mov_b32 a57, a121
-; CHECK-NEXT: v_accvgpr_mov_b32 a58, a122
-; CHECK-NEXT: v_accvgpr_mov_b32 a59, a123
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[96:111], v[30:31], v[30:31], a[96:111]
-; CHECK-NEXT: v_accvgpr_mov_b32 a60, a124
-; CHECK-NEXT: v_accvgpr_mov_b32 a61, a125
-; CHECK-NEXT: v_accvgpr_mov_b32 a62, a126
-; CHECK-NEXT: v_accvgpr_mov_b32 a63, a127
-; CHECK-NEXT: v_pk_mov_b32 v[24:25], v[226:227], v[226:227] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[26:27], v[228:229], v[228:229] op_sel:[0,1]
-; CHECK-NEXT: v_mov_b32_e32 v115, v3
-; CHECK-NEXT: v_mov_b32_e32 v116, v3
-; CHECK-NEXT: v_mov_b32_e32 v117, v3
-; CHECK-NEXT: v_mov_b32_e32 v118, v3
-; CHECK-NEXT: v_mov_b32_e32 v119, v3
-; CHECK-NEXT: v_mov_b32_e32 v120, v3
-; CHECK-NEXT: v_mov_b32_e32 v121, v3
-; CHECK-NEXT: v_mov_b32_e32 v122, v3
-; CHECK-NEXT: v_mov_b32_e32 v123, v3
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[80:95], v[30:31], v[30:31], a[64:79]
-; CHECK-NEXT: v_mov_b32_e32 v124, v3
-; CHECK-NEXT: v_mov_b32_e32 v125, v3
-; CHECK-NEXT: v_mov_b32_e32 v126, v3
-; CHECK-NEXT: v_mov_b32_e32 v127, v3
-; CHECK-NEXT: v_mov_b32_e32 v128, v3
-; CHECK-NEXT: v_mov_b32_e32 v129, v3
-; CHECK-NEXT: v_pk_mov_b32 v[34:35], v[214:215], v[214:215] op_sel:[0,1]
-; CHECK-NEXT: v_mov_b32_e32 v67, v66
-; CHECK-NEXT: v_mov_b32_e32 v243, v242
+; CHECK-NEXT: v_mov_b32_e32 v83, v3
+; CHECK-NEXT: v_mov_b32_e32 v84, v3
+; CHECK-NEXT: v_mov_b32_e32 v85, v3
+; CHECK-NEXT: v_mov_b32_e32 v86, v3
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[162:177], v[134:135], v[134:135], v[162:177]
+; CHECK-NEXT: v_mov_b32_e32 v87, v3
+; CHECK-NEXT: v_mov_b32_e32 v88, v3
+; CHECK-NEXT: v_mov_b32_e32 v89, v3
+; CHECK-NEXT: v_accvgpr_mov_b32 a36, a98
+; CHECK-NEXT: v_accvgpr_mov_b32 a37, a99
+; CHECK-NEXT: v_accvgpr_mov_b32 a38, a100
+; CHECK-NEXT: v_accvgpr_mov_b32 a39, a101
+; CHECK-NEXT: v_accvgpr_mov_b32 a40, a102
+; CHECK-NEXT: v_accvgpr_mov_b32 a41, a103
+; CHECK-NEXT: v_accvgpr_mov_b32 a42, a104
+; CHECK-NEXT: v_accvgpr_mov_b32 a43, a105
+; CHECK-NEXT: v_accvgpr_mov_b32 a44, a106
+; CHECK-NEXT: v_accvgpr_mov_b32 a45, a107
+; CHECK-NEXT: v_accvgpr_mov_b32 a46, a108
+; CHECK-NEXT: v_accvgpr_mov_b32 a47, a109
+; CHECK-NEXT: v_accvgpr_mov_b32 a48, a110
+; CHECK-NEXT: v_accvgpr_mov_b32 a49, a111
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[146:161], v[134:135], v[134:135], v[146:161]
+; CHECK-NEXT: v_mov_b32_e32 v90, v3
+; CHECK-NEXT: v_mov_b32_e32 v91, v3
+; CHECK-NEXT: v_mov_b32_e32 v92, v3
+; CHECK-NEXT: v_mov_b32_e32 v93, v3
+; CHECK-NEXT: v_mov_b32_e32 v94, v3
+; CHECK-NEXT: v_mov_b32_e32 v95, v3
+; CHECK-NEXT: v_mov_b32_e32 v96, v3
+; CHECK-NEXT: v_mov_b32_e32 v97, v3
+; CHECK-NEXT: v_pk_mov_b32 v[26:27], v[162:163], v[162:163] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[28:29], v[164:165], v[164:165] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[30:31], v[166:167], v[166:167] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[32:33], v[168:169], v[168:169] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[34:35], v[170:171], v[170:171] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[36:37], v[172:173], v[172:173] op_sel:[0,1]
+; CHECK-NEXT: v_pk_mov_b32 v[38:39], v[174:175], v[174:175] op_sel:[0,1]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[36:51], v[134:135], v[134:135], a[36:51]
+; CHECK-NEXT: v_pk_mov_b32 v[40:41], v[176:177], v[176:177] op_sel:[0,1]
; CHECK-NEXT: v_accvgpr_mov_b32 a17, a16
-; CHECK-NEXT: v_pk_mov_b32 v[36:37], v[216:217], v[216:217] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[38:39], v[218:219], v[218:219] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[40:41], v[220:221], v[220:221] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[42:43], v[222:223], v[222:223] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[44:45], v[224:225], v[224:225] op_sel:[0,1]
-; CHECK-NEXT: v_accvgpr_mov_b32 a79, a15
-; CHECK-NEXT: v_accvgpr_mov_b32 a78, a14
-; CHECK-NEXT: v_accvgpr_mov_b32 a77, a13
-; CHECK-NEXT: v_accvgpr_mov_b32 a76, a12
-; CHECK-NEXT: v_accvgpr_mov_b32 a75, a11
-; CHECK-NEXT: v_accvgpr_mov_b32 a74, a10
-; CHECK-NEXT: v_accvgpr_mov_b32 a73, a9
-; CHECK-NEXT: v_accvgpr_mov_b32 a72, a8
-; CHECK-NEXT: v_accvgpr_mov_b32 a71, a7
-; CHECK-NEXT: v_accvgpr_mov_b32 a70, a6
-; CHECK-NEXT: v_accvgpr_mov_b32 a69, a5
-; CHECK-NEXT: v_accvgpr_mov_b32 a68, a4
-; CHECK-NEXT: v_accvgpr_mov_b32 a67, a3
-; CHECK-NEXT: v_accvgpr_mov_b32 a66, a2
-; CHECK-NEXT: v_accvgpr_mov_b32 a65, a1
-; CHECK-NEXT: v_accvgpr_mov_b32 a64, a0
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[146:161], v[30:31], v[30:31], v[146:161]
-; CHECK-NEXT: v_accvgpr_mov_b32 a1, a0
-; CHECK-NEXT: v_pk_mov_b32 v[46:47], v[226:227], v[226:227] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[48:49], v[228:229], v[228:229] op_sel:[0,1]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[64:79], v[30:31], v[30:31], a[64:79]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[86:101], v[30:31], v[30:31], v[86:101]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[162:177], v[30:31], v[30:31], v[162:177]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[50:65], v[30:31], v[30:31], v[50:65]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[196:211], v[30:31], v[30:31], v[196:211]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[30:31], v[28:29], v[4:19]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[130:145], v[30:31], v[30:31], v[130:145]
+; CHECK-NEXT: v_mov_b32_e32 v243, v242
+; CHECK-NEXT: v_accvgpr_mov_b32 a33, a32
+; CHECK-NEXT: v_mov_b32_e32 v67, v66
+; CHECK-NEXT: v_mov_b32_e32 v2, v42
+; CHECK-NEXT: v_accvgpr_mov_b32 a0, a34
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[134:135], v[134:135], v[4:19]
+; CHECK-NEXT: s_nop 10
+; CHECK-NEXT: v_accvgpr_mov_b32 a32, a36
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[38:53], v[134:135], v[134:135], a[66:81]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[116:131], v[134:135], v[134:135], v[116:131]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[98:113], v[134:135], v[134:135], v[98:113]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[180:195], v[134:135], v[134:135], v[180:195]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[50:65], v[134:135], v[134:135], v[50:65]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[82:97], v[134:135], v[134:135], v[82:97]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[226:241], v[134:135], v[134:135], v[226:241]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[146:161], v[134:135], v[134:135], v[146:161]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[26:41], v[134:135], v[134:135], v[26:41]
; CHECK-NEXT: s_nop 15
; CHECK-NEXT: s_nop 1
-; CHECK-NEXT: v_pk_mov_b32 v[12:13], v[214:215], v[214:215] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[14:15], v[216:217], v[216:217] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[16:17], v[218:219], v[218:219] op_sel:[0,1]
-; CHECK-NEXT: v_pk_mov_b32 v[18:19], v[220:221], v[220:221] op_sel:[0,1]
-; CHECK-NEXT: v_mov_b32_e32 v66, v11
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[96:111], v[30:31], v[30:31], a[96:111]
-; CHECK-NEXT: v_mov_b32_e32 v145, v130
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[80:95], v[30:31], v[30:31], a[80:95]
-; CHECK-NEXT: s_nop 15
-; CHECK-NEXT: s_nop 0
-; CHECK-NEXT: v_accvgpr_read_b32 v2, a96
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[18:33], v[30:31], v[30:31], a[18:33]
-; CHECK-NEXT: v_accvgpr_read_b32 v70, a80
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[48:63], v[30:31], v[30:31], a[48:63]
+; CHECK-NEXT: v_mov_b32_e32 v161, v146
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[4:19], v[134:135], v[134:135], v[4:19]
+; CHECK-NEXT: v_mov_b32_e32 v163, v26
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[116:131], v[134:135], v[132:133], v[116:131]
; CHECK-NEXT: s_nop 15
; CHECK-NEXT: s_nop 0
-; CHECK-NEXT: v_accvgpr_read_b32 v212, a18
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[12:27], v[30:31], v[30:31], v[12:27]
-; CHECK-NEXT: v_accvgpr_mov_b32 a16, a48
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[146:161], v[30:31], v[30:31], v[146:161]
+; CHECK-NEXT: v_mov_b32_e32 v20, v4
+; CHECK-NEXT: v_accvgpr_read_b32 v18, a66
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[38:53], v[134:135], v[134:135], a[38:53]
+; CHECK-NEXT: v_mov_b32_e32 v66, v123
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[98:113], v[134:135], v[134:135], v[98:113]
; CHECK-NEXT: s_nop 15
; CHECK-NEXT: s_nop 0
-; CHECK-NEXT: v_mov_b32_e32 v35, v12
-; CHECK-NEXT: v_mov_b32_e32 v20, v68
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[86:101], v[30:31], v[30:31], v[86:101]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 a[64:79], v[30:31], v[30:31], a[64:79]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[162:177], v[30:31], v[30:31], v[162:177]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[50:65], v[30:31], v[30:31], v[50:65]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[196:211], v[30:31], v[30:31], v[196:211]
-; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[114:129], v[30:31], v[30:31], v[114:129]
+; CHECK-NEXT: v_accvgpr_read_b32 v22, a38
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[180:195], v[134:135], v[134:135], v[180:195]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[50:65], v[134:135], v[134:135], v[50:65]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[82:97], v[134:135], v[134:135], v[82:97]
+; CHECK-NEXT: v_mfma_f32_32x32x8f16 v[226:241], v[134:135], v[134:135], v[226:241]
; CHECK-NEXT: s_mov_b64 vcc, vcc
; CHECK-NEXT: s_cbranch_vccnz .LBB0_1
; CHECK-NEXT: ; %bb.2: ; %DummyReturnBlock
diff --git a/llvm/test/CodeGen/AMDGPU/sched-assert-dead-def-subreg-use-other-subreg.mir b/llvm/test/CodeGen/AMDGPU/sched-assert-dead-def-subreg-use-other-subreg.mir
index d511b47a4f864..2aaeb80a41d63 100644
--- a/llvm/test/CodeGen/AMDGPU/sched-assert-dead-def-subreg-use-other-subreg.mir
+++ b/llvm/test/CodeGen/AMDGPU/sched-assert-dead-def-subreg-use-other-subreg.mir
@@ -24,6 +24,7 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: undef [[V_MOV_B32_e32_:%[0-9]+]].sub3:vreg_512 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: [[V_MOV_B32_e32_1:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
+ ; CHECK-NEXT: [[V_ADD_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 0, [[V_MOV_B32_e32_1]], implicit $exec
; CHECK-NEXT: [[V_MOV_B32_e32_2:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vreg_512 = COPY [[V_MOV_B32_e32_]]
; CHECK-NEXT: {{ $}}
@@ -33,7 +34,6 @@ body: |
; CHECK-NEXT: BUFFER_STORE_DWORD_OFFEN [[V_MOV_B32_e32_]].sub3, undef %5:vgpr_32, $sgpr24_sgpr25_sgpr26_sgpr27, $sgpr32, 0, 0, 0, implicit $exec :: (store (s32), align 8, addrspace 5)
; CHECK-NEXT: dead [[DS_READ_B32_gfx9_:%[0-9]+]]:vgpr_32 = DS_READ_B32_gfx9 undef %7:vgpr_32, 0, 0, implicit $exec
; CHECK-NEXT: dead [[DS_READ_B64_gfx9_:%[0-9]+]]:vreg_64 = DS_READ_B64_gfx9 [[V_MOV_B32_e32_1]], 0, 0, implicit $exec
- ; CHECK-NEXT: [[V_ADD_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 0, [[V_MOV_B32_e32_1]], implicit $exec
; CHECK-NEXT: dead [[DS_READ_B128_gfx9_:%[0-9]+]]:vreg_128 = DS_READ_B128_gfx9 [[V_ADD_U32_e32_]], 0, 0, implicit $exec
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY [[COPY]].sub0
; CHECK-NEXT: undef [[COPY2:%[0-9]+]].sub1:vreg_512 = COPY [[COPY]].sub1
diff --git a/llvm/test/CodeGen/AMDGPU/sched-dynamic-vgpr-blocks.mir b/llvm/test/CodeGen/AMDGPU/sched-dynamic-vgpr-blocks.mir
index a103a039370a1..3f29d2c09434e 100644
--- a/llvm/test/CodeGen/AMDGPU/sched-dynamic-vgpr-blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/sched-dynamic-vgpr-blocks.mir
@@ -4,7 +4,7 @@
# In dynamic VGPR mode we don't want to waste VGPR blocks, so we revert any
# schedule that increases the number of blocks we need to allocate. The
# scheduler will try to hoist the loads above the dependent adds, which will
-# increase the pressure in the region from 5 to 18 VGPRs, i.e. from one block of
+# increase the pressure in the region from 4 to 17 VGPRs, i.e. from one block of
# 16 to 2. This should trigger the code that performs the revert based on the
# number of VGPR blocks.
# The -amdgpu-stress-vgpr flag is only there to keep the test small (the
@@ -15,8 +15,8 @@
attributes #0 = { "amdgpu-dynamic-vgpr-block-size"="16" }
...
---
-# DEBUG: Region register pressure: VGPRs: 5
-# DEBUG: Pressure after scheduling: VGPRs: 18
+# DEBUG: Region register pressure: VGPRs: 4
+# DEBUG: Pressure after scheduling: VGPRs: 17
# DEBUG-NEXT: Region: 0
# DEBUG-NEXT: Occupancy before scheduling: 16, after 16.
# DEBUG-NEXT: Reverting scheduling for region 0
diff --git a/llvm/test/CodeGen/AMDGPU/sched_mfma_rewrite_copies.mir b/llvm/test/CodeGen/AMDGPU/sched_mfma_rewrite_copies.mir
index a4a94089d2f7b..2890969057ba0 100644
--- a/llvm/test/CodeGen/AMDGPU/sched_mfma_rewrite_copies.mir
+++ b/llvm/test/CodeGen/AMDGPU/sched_mfma_rewrite_copies.mir
@@ -5280,11 +5280,11 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: [[DEF13:%[0-9]+]]:av_128_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF14:%[0-9]+]]:av_128_align2 = IMPLICIT_DEF
- ; CHECK-NEXT: [[DEF15:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
; CHECK-NEXT: undef [[AV_MOV_:%[0-9]+]].sub0:vreg_128_align2 = AV_MOV_B32_IMM_PSEUDO 0, implicit $exec
; CHECK-NEXT: [[AV_MOV_:%[0-9]+]].sub1:vreg_128_align2 = AV_MOV_B32_IMM_PSEUDO 0, implicit $exec
; CHECK-NEXT: [[AV_MOV_:%[0-9]+]].sub2:vreg_128_align2 = AV_MOV_B32_IMM_PSEUDO 0, implicit $exec
; CHECK-NEXT: [[AV_MOV_:%[0-9]+]].sub3:vreg_128_align2 = AV_MOV_B32_IMM_PSEUDO 0, implicit $exec
+ ; CHECK-NEXT: [[DEF15:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF13]], [[DEF14]], [[AV_MOV_]], 4, 4, [[DEF15]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_1:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF13]], [[DEF14]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_]], 4, 4, [[DEF15]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_2:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF13]], [[DEF14]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_1]], 4, 4, [[DEF15]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
@@ -5380,9 +5380,9 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: [[DEF13:%[0-9]+]]:av_128_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF14:%[0-9]+]]:av_128_align2 = IMPLICIT_DEF
- ; CHECK-NEXT: [[DEF15:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
; CHECK-NEXT: undef [[AV_MOV_:%[0-9]+]].sub0_sub1:vreg_128_align2 = AV_MOV_B64_IMM_PSEUDO 0, implicit $exec
; CHECK-NEXT: [[AV_MOV_:%[0-9]+]].sub2_sub3:vreg_128_align2 = AV_MOV_B64_IMM_PSEUDO 0, implicit $exec
+ ; CHECK-NEXT: [[DEF15:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF13]], [[DEF14]], [[AV_MOV_]], 4, 4, [[DEF15]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_1:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF13]], [[DEF14]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_]], 4, 4, [[DEF15]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_2:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF13]], [[DEF14]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_1]], 4, 4, [[DEF15]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
@@ -5479,18 +5479,16 @@ body: |
; CHECK-NEXT: [[DEF15:%[0-9]+]]:av_128_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF16:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF14]], [[DEF15]], [[DEF13]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:areg_128_align2 = COPY [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_]]
- ; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64_:%[0-9]+]]:areg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64 [[DEF14]], [[DEF15]], [[COPY]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64_1:%[0-9]+]]:areg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64 [[DEF14]], [[DEF15]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64_]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64_2:%[0-9]+]]:areg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64 [[DEF14]], [[DEF15]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64_1]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_1:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF14]], [[DEF15]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_2:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF14]], [[DEF15]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_1]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_3:%[0-9]+]]:vreg_128_align2 = contract nofpexcept V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64 [[DEF14]], [[DEF15]], [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_2]], 4, 4, [[DEF16]].sub0, [[DEF12]], 0, 0, implicit $mode, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: undef [[V_ADD_U32_e32_:%[0-9]+]].sub0:vreg_128_align2 = V_ADD_U32_e32 [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_]].sub0, [[DEF12]], implicit $exec
- ; CHECK-NEXT: [[COPY1:%[0-9]+]]:vreg_128_align2 = COPY [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_e64_2]]
- ; CHECK-NEXT: [[COPY2:%[0-9]+]]:vreg_128_align2 = COPY [[COPY1]]
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:vreg_128_align2 = COPY [[V_MFMA_SCALE_F32_16X16X128_F8F6F4_f4_f4_vgprcd_e64_3]]
; CHECK-NEXT: SCHED_BARRIER 0
; CHECK-NEXT: [[DEF17:%[0-9]+]]:vreg_1024 = IMPLICIT_DEF
- ; CHECK-NEXT: KILL [[DEF17]], [[DEF]], [[DEF1]], [[DEF2]], [[DEF3]], [[DEF4]], [[DEF5]], [[DEF6]], [[DEF7]], [[DEF8]], [[DEF9]], [[DEF10]], [[DEF11]], [[DEF13]], [[V_ADD_U32_e32_]], [[COPY2]]
+ ; CHECK-NEXT: KILL [[DEF17]], [[DEF]], [[DEF1]], [[DEF2]], [[DEF3]], [[DEF4]], [[DEF5]], [[DEF6]], [[DEF7]], [[DEF8]], [[DEF9]], [[DEF10]], [[DEF11]], [[DEF13]], [[V_ADD_U32_e32_]], [[COPY]]
; CHECK-NEXT: S_NOP 0, implicit %12, implicit %13
; CHECK-NEXT: S_ENDPGM 0
bb.0:
@@ -5564,26 +5562,24 @@ body: |
; CHECK-NEXT: [[DEF7:%[0-9]+]]:vreg_1024 = IMPLICIT_DEF
; CHECK-NEXT: SCHED_BARRIER 0
; CHECK-NEXT: [[AV_MOV_:%[0-9]+]]:vreg_64_align2 = AV_MOV_B64_IMM_PSEUDO 0, implicit $exec
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:areg_64_align2 = COPY [[AV_MOV_]]
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.2(0x80000000)
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: [[DEF8:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
; CHECK-NEXT: [[DEF9:%[0-9]+]]:vreg_64_align2 = IMPLICIT_DEF
- ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_e64_:%[0-9]+]]:areg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_e64 [[DEF8]], [[DEF9]], [[COPY]], 0, 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_e64_1:%[0-9]+]]:areg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_e64 [[DEF8]], [[DEF9]], [[V_MFMA_F64_4X4X4F64_e64_]], 0, 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_e64_2:%[0-9]+]]:areg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_e64 [[DEF8]], [[DEF9]], [[V_MFMA_F64_4X4X4F64_e64_1]], 0, 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_e64_3:%[0-9]+]]:areg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_e64 [[DEF8]], [[DEF9]], [[V_MFMA_F64_4X4X4F64_e64_2]], 0, 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_vgprcd_e64_:%[0-9]+]]:vreg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_vgprcd_e64 [[DEF8]], [[DEF9]], [[AV_MOV_]], 0, 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_vgprcd_e64_1:%[0-9]+]]:vreg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_vgprcd_e64 [[DEF8]], [[DEF9]], [[V_MFMA_F64_4X4X4F64_vgprcd_e64_]], 0, 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_vgprcd_e64_2:%[0-9]+]]:vreg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_vgprcd_e64 [[DEF8]], [[DEF9]], [[V_MFMA_F64_4X4X4F64_vgprcd_e64_1]], 0, 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_MFMA_F64_4X4X4F64_vgprcd_e64_3:%[0-9]+]]:vreg_64_align2 = contract nofpexcept V_MFMA_F64_4X4X4F64_vgprcd_e64 [[DEF8]], [[DEF9]], [[V_MFMA_F64_4X4X4F64_vgprcd_e64_2]], 0, 0, 0, implicit $mode, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: [[DEF10:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; CHECK-NEXT: undef [[V_ADD_U32_e32_:%[0-9]+]].sub0:vreg_64_align2 = V_ADD_U32_e32 [[AV_MOV_]].sub0, [[DEF10]], implicit $exec
- ; CHECK-NEXT: [[COPY1:%[0-9]+]]:vreg_64_align2 = COPY [[V_MFMA_F64_4X4X4F64_e64_3]]
- ; CHECK-NEXT: [[COPY2:%[0-9]+]]:vreg_64_align2 = COPY [[COPY1]]
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:vreg_64_align2 = COPY [[V_MFMA_F64_4X4X4F64_vgprcd_e64_3]]
; CHECK-NEXT: SCHED_BARRIER 0
; CHECK-NEXT: [[DEF11:%[0-9]+]]:vreg_1024 = IMPLICIT_DEF
- ; CHECK-NEXT: KILL [[DEF11]], [[DEF]], [[DEF1]], [[DEF2]], [[DEF3]], [[DEF4]], [[DEF5]], [[DEF6]], [[DEF7]], [[V_ADD_U32_e32_]], [[COPY2]]
+ ; CHECK-NEXT: KILL [[DEF11]], [[DEF]], [[DEF1]], [[DEF2]], [[DEF3]], [[DEF4]], [[DEF5]], [[DEF6]], [[DEF7]], [[V_ADD_U32_e32_]], [[COPY]]
; CHECK-NEXT: S_NOP 0, implicit %9, implicit %10
; CHECK-NEXT: S_ENDPGM 0
bb.0:
diff --git a/llvm/test/CodeGen/AMDGPU/schedule-regpressure-ilp-metric-spills.mir b/llvm/test/CodeGen/AMDGPU/schedule-regpressure-ilp-metric-spills.mir
index 6d9f8b6b3c2c5..69dc45141d89f 100644
--- a/llvm/test/CodeGen/AMDGPU/schedule-regpressure-ilp-metric-spills.mir
+++ b/llvm/test/CodeGen/AMDGPU/schedule-regpressure-ilp-metric-spills.mir
@@ -13,13 +13,10 @@
# GCN-NOT: SI_SPILL_
# GCN: S_ENDPGM
+# The GCN trackers used to over-estimate pressure here and spill; they now
+# agree with the generic tracker that no spilling is needed.
# GCN-GCNTRACKER-LABEL: name: no_sched_metric_due_to_spills
-# GCN-GCNTRACKER: SI_SPILL_V32_SAVE
-# GCN-GCNTRACKER: SI_SPILL_V32_SAVE
-# GCN-GCNTRACKER: SI_SPILL_V32_SAVE
-# GCN-GCNTRACKER: SI_SPILL_V32_SAVE
-# GCN-GCNTRACKER: SI_SPILL_V32_SAVE
-# GCN-GCNTRACKER: SI_SPILL_V32_SAVE
+# GCN-GCNTRACKER-NOT: SI_SPILL_
# GCN-GCNTRACKER: S_ENDPGM
# When using the GCN Trackers, the scheduler is able to acieve desired occupancy without running high-RP-reschedule stage. However, the RP is still high,
diff --git a/llvm/test/CodeGen/AMDGPU/spill-agpr.ll b/llvm/test/CodeGen/AMDGPU/spill-agpr.ll
index 1ca643d90aac0..20c63eeb2f220 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-agpr.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-agpr.ll
@@ -433,12 +433,12 @@ define amdgpu_kernel void @max_6regs_used_8a(ptr addrspace(1) %arg) #4 {
; GFX90A-NEXT: ;;#ASMSTART
; GFX90A-NEXT: ; def a[0:3]
; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: v_lshlrev_b32_e32 v0, 4, v0
; GFX90A-NEXT: buffer_store_dword a0, off, s[8:11], 0 ; 4-byte Folded Spill
; GFX90A-NEXT: s_nop 0
; GFX90A-NEXT: buffer_store_dword a1, off, s[8:11], 0 offset:4 ; 4-byte Folded Spill
; GFX90A-NEXT: buffer_store_dword a2, off, s[8:11], 0 offset:8 ; 4-byte Folded Spill
; GFX90A-NEXT: buffer_store_dword a3, off, s[8:11], 0 offset:12 ; 4-byte Folded Spill
-; GFX90A-NEXT: v_lshlrev_b32_e32 v0, 4, v0
; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
; GFX90A-NEXT: global_load_dwordx4 a[0:3], v0, s[2:3]
; GFX90A-NEXT: v_mov_b32_e32 v2, 1.0
diff --git a/llvm/test/CodeGen/AMDGPU/uaddsat.ll b/llvm/test/CodeGen/AMDGPU/uaddsat.ll
index e188bb032b029..213eff5a42dc2 100644
--- a/llvm/test/CodeGen/AMDGPU/uaddsat.ll
+++ b/llvm/test/CodeGen/AMDGPU/uaddsat.ll
@@ -836,7 +836,6 @@ define <16 x i32> @v_uaddsat_v16i32(<16 x i32> %lhs, <16 x i32> %rhs) {
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX6-NEXT: v_not_b32_e32 v31, v16
; GFX6-NEXT: v_min_u32_e32 v0, v0, v31
-; GFX6-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX6-NEXT: v_add_i32_e32 v0, vcc, v0, v16
; GFX6-NEXT: v_not_b32_e32 v16, v17
; GFX6-NEXT: v_min_u32_e32 v1, v1, v16
@@ -846,27 +845,28 @@ define <16 x i32> @v_uaddsat_v16i32(<16 x i32> %lhs, <16 x i32> %rhs) {
; GFX6-NEXT: v_min_u32_e32 v3, v3, v16
; GFX6-NEXT: v_not_b32_e32 v16, v20
; GFX6-NEXT: v_min_u32_e32 v4, v4, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v21
-; GFX6-NEXT: v_min_u32_e32 v5, v5, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v22
-; GFX6-NEXT: v_min_u32_e32 v6, v6, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v23
-; GFX6-NEXT: v_min_u32_e32 v7, v7, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v24
-; GFX6-NEXT: v_min_u32_e32 v8, v8, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v25
-; GFX6-NEXT: v_min_u32_e32 v9, v9, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v26
-; GFX6-NEXT: v_min_u32_e32 v10, v10, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v27
-; GFX6-NEXT: v_min_u32_e32 v11, v11, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v28
-; GFX6-NEXT: v_min_u32_e32 v12, v12, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v29
-; GFX6-NEXT: v_min_u32_e32 v13, v13, v16
-; GFX6-NEXT: v_not_b32_e32 v16, v30
-; GFX6-NEXT: v_min_u32_e32 v14, v14, v16
+; GFX6-NEXT: buffer_load_dword v16, off, s[0:3], s32
; GFX6-NEXT: v_add_i32_e32 v1, vcc, v1, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v21
+; GFX6-NEXT: v_min_u32_e32 v5, v5, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v22
+; GFX6-NEXT: v_min_u32_e32 v6, v6, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v23
+; GFX6-NEXT: v_min_u32_e32 v7, v7, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v24
+; GFX6-NEXT: v_min_u32_e32 v8, v8, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v25
+; GFX6-NEXT: v_min_u32_e32 v9, v9, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v26
+; GFX6-NEXT: v_min_u32_e32 v10, v10, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v27
+; GFX6-NEXT: v_min_u32_e32 v11, v11, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v28
+; GFX6-NEXT: v_min_u32_e32 v12, v12, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v29
+; GFX6-NEXT: v_min_u32_e32 v13, v13, v17
+; GFX6-NEXT: v_not_b32_e32 v17, v30
+; GFX6-NEXT: v_min_u32_e32 v14, v14, v17
; GFX6-NEXT: v_add_i32_e32 v2, vcc, v2, v18
; GFX6-NEXT: v_add_i32_e32 v3, vcc, v3, v19
; GFX6-NEXT: v_add_i32_e32 v4, vcc, v4, v20
@@ -881,9 +881,9 @@ define <16 x i32> @v_uaddsat_v16i32(<16 x i32> %lhs, <16 x i32> %rhs) {
; GFX6-NEXT: v_add_i32_e32 v13, vcc, v13, v29
; GFX6-NEXT: v_add_i32_e32 v14, vcc, v14, v30
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: v_not_b32_e32 v16, v31
-; GFX6-NEXT: v_min_u32_e32 v15, v15, v16
-; GFX6-NEXT: v_add_i32_e32 v15, vcc, v15, v31
+; GFX6-NEXT: v_not_b32_e32 v17, v16
+; GFX6-NEXT: v_min_u32_e32 v15, v15, v17
+; GFX6-NEXT: v_add_i32_e32 v15, vcc, v15, v16
; GFX6-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: v_uaddsat_v16i32:
diff --git a/llvm/test/CodeGen/AMDGPU/vector-reduce-fmax.ll b/llvm/test/CodeGen/AMDGPU/vector-reduce-fmax.ll
index e0c5f653274a5..ae8dbee4391f7 100644
--- a/llvm/test/CodeGen/AMDGPU/vector-reduce-fmax.ll
+++ b/llvm/test/CodeGen/AMDGPU/vector-reduce-fmax.ll
@@ -3514,33 +3514,33 @@ define double @test_vector_reduce_fmax_v16double(<16 x double> %v) {
; GFX7-GISEL-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[0:1]
; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[2:3], v[2:3]
-; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[4:5], v[4:5]
-; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[6:7], v[6:7]
-; GFX7-GISEL-NEXT: v_max_f64 v[8:9], v[8:9], v[8:9]
-; GFX7-GISEL-NEXT: v_max_f64 v[10:11], v[10:11], v[10:11]
-; GFX7-GISEL-NEXT: v_max_f64 v[12:13], v[12:13], v[12:13]
-; GFX7-GISEL-NEXT: v_max_f64 v[14:15], v[14:15], v[14:15]
-; GFX7-GISEL-NEXT: v_max_f64 v[16:17], v[16:17], v[16:17]
-; GFX7-GISEL-NEXT: v_max_f64 v[18:19], v[18:19], v[18:19]
-; GFX7-GISEL-NEXT: v_max_f64 v[20:21], v[20:21], v[20:21]
-; GFX7-GISEL-NEXT: v_max_f64 v[22:23], v[22:23], v[22:23]
-; GFX7-GISEL-NEXT: v_max_f64 v[24:25], v[24:25], v[24:25]
-; GFX7-GISEL-NEXT: v_max_f64 v[26:27], v[26:27], v[26:27]
-; GFX7-GISEL-NEXT: v_max_f64 v[28:29], v[28:29], v[28:29]
; GFX7-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
-; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[6:7]
-; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[10:11]
-; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[12:13], v[14:15]
-; GFX7-GISEL-NEXT: v_max_f64 v[8:9], v[16:17], v[18:19]
-; GFX7-GISEL-NEXT: v_max_f64 v[10:11], v[20:21], v[22:23]
-; GFX7-GISEL-NEXT: v_max_f64 v[12:13], v[24:25], v[26:27]
+; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[4:5]
+; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[6:7], v[6:7]
+; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[10:11], v[10:11]
+; GFX7-GISEL-NEXT: v_max_f64 v[10:11], v[14:15], v[14:15]
+; GFX7-GISEL-NEXT: v_max_f64 v[14:15], v[18:19], v[18:19]
+; GFX7-GISEL-NEXT: v_max_f64 v[18:19], v[22:23], v[22:23]
+; GFX7-GISEL-NEXT: v_max_f64 v[22:23], v[26:27], v[26:27]
+; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[2:3], v[4:5]
+; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[8:9]
+; GFX7-GISEL-NEXT: v_max_f64 v[8:9], v[12:13], v[12:13]
+; GFX7-GISEL-NEXT: v_max_f64 v[12:13], v[16:17], v[16:17]
+; GFX7-GISEL-NEXT: v_max_f64 v[16:17], v[20:21], v[20:21]
+; GFX7-GISEL-NEXT: v_max_f64 v[20:21], v[24:25], v[24:25]
+; GFX7-GISEL-NEXT: v_max_f64 v[24:25], v[28:29], v[28:29]
; GFX7-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
+; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[4:5], v[6:7]
+; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[8:9], v[10:11]
+; GFX7-GISEL-NEXT: v_max_f64 v[8:9], v[12:13], v[14:15]
+; GFX7-GISEL-NEXT: v_max_f64 v[10:11], v[16:17], v[18:19]
+; GFX7-GISEL-NEXT: v_max_f64 v[12:13], v[20:21], v[22:23]
; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[6:7]
; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[10:11]
; GFX7-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(0)
-; GFX7-GISEL-NEXT: v_max_f64 v[30:31], v[30:31], v[30:31]
-; GFX7-GISEL-NEXT: v_max_f64 v[14:15], v[28:29], v[30:31]
+; GFX7-GISEL-NEXT: v_max_f64 v[26:27], v[30:31], v[30:31]
+; GFX7-GISEL-NEXT: v_max_f64 v[14:15], v[24:25], v[26:27]
; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[12:13], v[14:15]
; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[6:7]
; GFX7-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
@@ -3590,33 +3590,33 @@ define double @test_vector_reduce_fmax_v16double(<16 x double> %v) {
; GFX8-GISEL-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX8-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[0:1]
; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[2:3], v[2:3]
-; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[4:5], v[4:5]
-; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[6:7], v[6:7]
-; GFX8-GISEL-NEXT: v_max_f64 v[8:9], v[8:9], v[8:9]
-; GFX8-GISEL-NEXT: v_max_f64 v[10:11], v[10:11], v[10:11]
-; GFX8-GISEL-NEXT: v_max_f64 v[12:13], v[12:13], v[12:13]
-; GFX8-GISEL-NEXT: v_max_f64 v[14:15], v[14:15], v[14:15]
-; GFX8-GISEL-NEXT: v_max_f64 v[16:17], v[16:17], v[16:17]
-; GFX8-GISEL-NEXT: v_max_f64 v[18:19], v[18:19], v[18:19]
-; GFX8-GISEL-NEXT: v_max_f64 v[20:21], v[20:21], v[20:21]
-; GFX8-GISEL-NEXT: v_max_f64 v[22:23], v[22:23], v[22:23]
-; GFX8-GISEL-NEXT: v_max_f64 v[24:25], v[24:25], v[24:25]
-; GFX8-GISEL-NEXT: v_max_f64 v[26:27], v[26:27], v[26:27]
-; GFX8-GISEL-NEXT: v_max_f64 v[28:29], v[28:29], v[28:29]
; GFX8-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
-; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[6:7]
-; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[10:11]
-; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[12:13], v[14:15]
-; GFX8-GISEL-NEXT: v_max_f64 v[8:9], v[16:17], v[18:19]
-; GFX8-GISEL-NEXT: v_max_f64 v[10:11], v[20:21], v[22:23]
-; GFX8-GISEL-NEXT: v_max_f64 v[12:13], v[24:25], v[26:27]
+; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[4:5]
+; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[6:7], v[6:7]
+; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[10:11], v[10:11]
+; GFX8-GISEL-NEXT: v_max_f64 v[10:11], v[14:15], v[14:15]
+; GFX8-GISEL-NEXT: v_max_f64 v[14:15], v[18:19], v[18:19]
+; GFX8-GISEL-NEXT: v_max_f64 v[18:19], v[22:23], v[22:23]
+; GFX8-GISEL-NEXT: v_max_f64 v[22:23], v[26:27], v[26:27]
+; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[2:3], v[4:5]
+; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[8:9]
+; GFX8-GISEL-NEXT: v_max_f64 v[8:9], v[12:13], v[12:13]
+; GFX8-GISEL-NEXT: v_max_f64 v[12:13], v[16:17], v[16:17]
+; GFX8-GISEL-NEXT: v_max_f64 v[16:17], v[20:21], v[20:21]
+; GFX8-GISEL-NEXT: v_max_f64 v[20:21], v[24:25], v[24:25]
+; GFX8-GISEL-NEXT: v_max_f64 v[24:25], v[28:29], v[28:29]
; GFX8-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
+; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[4:5], v[6:7]
+; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[8:9], v[10:11]
+; GFX8-GISEL-NEXT: v_max_f64 v[8:9], v[12:13], v[14:15]
+; GFX8-GISEL-NEXT: v_max_f64 v[10:11], v[16:17], v[18:19]
+; GFX8-GISEL-NEXT: v_max_f64 v[12:13], v[20:21], v[22:23]
; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[6:7]
; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[10:11]
; GFX8-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
-; GFX8-GISEL-NEXT: v_max_f64 v[30:31], v[30:31], v[30:31]
-; GFX8-GISEL-NEXT: v_max_f64 v[14:15], v[28:29], v[30:31]
+; GFX8-GISEL-NEXT: v_max_f64 v[26:27], v[30:31], v[30:31]
+; GFX8-GISEL-NEXT: v_max_f64 v[14:15], v[24:25], v[26:27]
; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[12:13], v[14:15]
; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[6:7]
; GFX8-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
diff --git a/llvm/test/CodeGen/AMDGPU/vector-reduce-fmaximum.ll b/llvm/test/CodeGen/AMDGPU/vector-reduce-fmaximum.ll
index 93c5ddad7fd21..da01970f46621 100644
--- a/llvm/test/CodeGen/AMDGPU/vector-reduce-fmaximum.ll
+++ b/llvm/test/CodeGen/AMDGPU/vector-reduce-fmaximum.ll
@@ -2406,136 +2406,136 @@ define double @test_vector_reduce_fmaximum_v16double(<16 x double> %v) {
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX7-NEXT: v_max_f64 v[32:33], v[8:9], v[24:25]
; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[8:9], v[24:25]
-; GFX7-NEXT: v_max_f64 v[8:9], v[0:1], v[16:17]
+; GFX7-NEXT: v_max_f64 v[8:9], v[8:9], v[24:25]
; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[16:17]
-; GFX7-NEXT: v_max_f64 v[0:1], v[12:13], v[28:29]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[12:13], v[28:29]
-; GFX7-NEXT: v_max_f64 v[12:13], v[10:11], v[26:27]
+; GFX7-NEXT: v_max_f64 v[24:25], v[0:1], v[16:17]
+; GFX7-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[10:11], v[26:27]
-; GFX7-NEXT: v_max_f64 v[10:11], v[2:3], v[18:19]
; GFX7-NEXT: v_cmp_u_f64_e64 s[10:11], v[2:3], v[18:19]
-; GFX7-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX7-NEXT: v_max_f64 v[2:3], v[4:5], v[20:21]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[12:13], v[4:5], v[20:21]
-; GFX7-NEXT: v_max_f64 v[4:5], v[6:7], v[22:23]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[14:15], v[6:7], v[22:23]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v13, v34, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v12, 0, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v11, v11, v34, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e32 v13, v33, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v12, v32, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v9, v9, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[4:5]
-; GFX7-NEXT: v_max_f64 v[16:17], v[10:11], v[6:7]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[10:11], v[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v34, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v5, v34, s[14:15]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[14:15]
-; GFX7-NEXT: v_max_f64 v[10:11], v[2:3], v[0:1]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[2:3], v[0:1]
-; GFX7-NEXT: v_max_f64 v[0:1], v[8:9], v[12:13]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[8:9], v[12:13]
-; GFX7-NEXT: v_cndmask_b32_e32 v3, v17, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v16, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[8:9]
+; GFX7-NEXT: v_max_f64 v[2:3], v[2:3], v[18:19]
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v9, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
+; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[12:13], v[28:29]
+; GFX7-NEXT: v_max_f64 v[8:9], v[12:13], v[28:29]
+; GFX7-NEXT: v_max_f64 v[12:13], v[4:5], v[20:21]
+; GFX7-NEXT: v_max_f64 v[20:21], v[10:11], v[26:27]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v25, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v24, 0, s[4:5]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[6:7], v[22:23]
+; GFX7-NEXT: v_cndmask_b32_e32 v9, v9, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v11, v13, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v10, v12, 0, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v13, v21, v16, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v12, v20, 0, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v8, v8, 0, vcc
+; GFX7-NEXT: v_max_f64 v[6:7], v[6:7], v[22:23]
+; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[12:13]
+; GFX7-NEXT: v_max_f64 v[2:3], v[2:3], v[12:13]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[4:5], v[0:1]
+; GFX7-NEXT: v_max_f64 v[0:1], v[4:5], v[0:1]
+; GFX7-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[10:11], v[8:9]
+; GFX7-NEXT: v_max_f64 v[8:9], v[10:11], v[8:9]
+; GFX7-NEXT: v_cndmask_b32_e32 v3, v3, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v1, v1, v16, s[8:9]
; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[8:9]
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: v_max_f64 v[6:7], v[14:15], v[30:31]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[14:15], v[30:31]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v7, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
-; GFX7-NEXT: v_max_f64 v[8:9], v[4:5], v[6:7]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[4:5], v[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v11, v34, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v10, 0, s[6:7]
-; GFX7-NEXT: v_max_f64 v[6:7], v[0:1], v[4:5]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[4:5]
-; GFX7-NEXT: v_cndmask_b32_e32 v1, v9, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
-; GFX7-NEXT: v_max_f64 v[4:5], v[2:3], v[0:1]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[0:1]
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v7, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v6, 0, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e32 v3, v5, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v4, 0, vcc
-; GFX7-NEXT: v_max_f64 v[4:5], v[0:1], v[2:3]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[14:15], v[30:31]
+; GFX7-NEXT: v_max_f64 v[12:13], v[14:15], v[30:31]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v13, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v12, 0, s[6:7]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[6:7], v[4:5]
+; GFX7-NEXT: v_max_f64 v[4:5], v[6:7], v[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v7, v9, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v6, v8, 0, s[4:5]
+; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[6:7]
+; GFX7-NEXT: v_max_f64 v[0:1], v[0:1], v[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[4:5]
+; GFX7-NEXT: v_max_f64 v[2:3], v[2:3], v[4:5]
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[2:3]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v4, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e32 v1, v5, v34, vcc
+; GFX7-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: test_vector_reduce_fmaximum_v16double:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX8-NEXT: v_max_f64 v[32:33], v[8:9], v[24:25]
; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[8:9], v[24:25]
-; GFX8-NEXT: v_max_f64 v[8:9], v[0:1], v[16:17]
+; GFX8-NEXT: v_max_f64 v[8:9], v[8:9], v[24:25]
; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[16:17]
-; GFX8-NEXT: v_max_f64 v[0:1], v[12:13], v[28:29]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[12:13], v[28:29]
-; GFX8-NEXT: v_max_f64 v[12:13], v[10:11], v[26:27]
+; GFX8-NEXT: v_max_f64 v[24:25], v[0:1], v[16:17]
+; GFX8-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[10:11], v[26:27]
-; GFX8-NEXT: v_max_f64 v[10:11], v[2:3], v[18:19]
; GFX8-NEXT: v_cmp_u_f64_e64 s[10:11], v[2:3], v[18:19]
-; GFX8-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX8-NEXT: v_max_f64 v[2:3], v[4:5], v[20:21]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[12:13], v[4:5], v[20:21]
-; GFX8-NEXT: v_max_f64 v[4:5], v[6:7], v[22:23]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[14:15], v[6:7], v[22:23]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v13, v34, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v12, 0, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v11, v34, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e32 v13, v33, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v12, v32, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v9, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[4:5]
-; GFX8-NEXT: v_max_f64 v[16:17], v[10:11], v[6:7]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[10:11], v[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v34, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v5, v34, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[14:15]
-; GFX8-NEXT: v_max_f64 v[10:11], v[2:3], v[0:1]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[2:3], v[0:1]
-; GFX8-NEXT: v_max_f64 v[0:1], v[8:9], v[12:13]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[8:9], v[12:13]
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v17, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v16, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[8:9]
+; GFX8-NEXT: v_max_f64 v[2:3], v[2:3], v[18:19]
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v9, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
+; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[12:13], v[28:29]
+; GFX8-NEXT: v_max_f64 v[8:9], v[12:13], v[28:29]
+; GFX8-NEXT: v_max_f64 v[12:13], v[4:5], v[20:21]
+; GFX8-NEXT: v_max_f64 v[20:21], v[10:11], v[26:27]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v25, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v24, 0, s[4:5]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[6:7], v[22:23]
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v9, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v11, v13, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v10, v12, 0, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v13, v21, v16, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v12, v20, 0, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, 0, vcc
+; GFX8-NEXT: v_max_f64 v[6:7], v[6:7], v[22:23]
+; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[12:13]
+; GFX8-NEXT: v_max_f64 v[2:3], v[2:3], v[12:13]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[4:5], v[0:1]
+; GFX8-NEXT: v_max_f64 v[0:1], v[4:5], v[0:1]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[10:11], v[8:9]
+; GFX8-NEXT: v_max_f64 v[8:9], v[10:11], v[8:9]
+; GFX8-NEXT: v_cndmask_b32_e32 v3, v3, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v16, s[8:9]
; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[8:9]
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_max_f64 v[6:7], v[14:15], v[30:31]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[14:15], v[30:31]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
-; GFX8-NEXT: v_max_f64 v[8:9], v[4:5], v[6:7]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[4:5], v[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v11, v34, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v10, 0, s[6:7]
-; GFX8-NEXT: v_max_f64 v[6:7], v[0:1], v[4:5]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v9, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
-; GFX8-NEXT: v_max_f64 v[4:5], v[2:3], v[0:1]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[0:1]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v7, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v6, 0, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v5, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v4, 0, vcc
-; GFX8-NEXT: v_max_f64 v[4:5], v[0:1], v[2:3]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[14:15], v[30:31]
+; GFX8-NEXT: v_max_f64 v[12:13], v[14:15], v[30:31]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v13, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v12, 0, s[6:7]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[6:7], v[4:5]
+; GFX8-NEXT: v_max_f64 v[4:5], v[6:7], v[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v9, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v8, 0, s[4:5]
+; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[6:7]
+; GFX8-NEXT: v_max_f64 v[0:1], v[0:1], v[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[4:5]
+; GFX8-NEXT: v_max_f64 v[2:3], v[2:3], v[4:5]
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[2:3]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v4, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v5, v34, vcc
+; GFX8-NEXT: v_max_f64 v[0:1], v[0:1], v[2:3]
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX9-LABEL: test_vector_reduce_fmaximum_v16double:
diff --git a/llvm/test/CodeGen/AMDGPU/vector-reduce-fmin.ll b/llvm/test/CodeGen/AMDGPU/vector-reduce-fmin.ll
index 055685c83bcbd..29b1e2d11c65c 100644
--- a/llvm/test/CodeGen/AMDGPU/vector-reduce-fmin.ll
+++ b/llvm/test/CodeGen/AMDGPU/vector-reduce-fmin.ll
@@ -3513,33 +3513,33 @@ define double @test_vector_reduce_fmin_v16double(<16 x double> %v) {
; GFX7-GISEL-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[0:1]
; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[2:3], v[2:3]
-; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[4:5], v[4:5]
-; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[6:7], v[6:7]
-; GFX7-GISEL-NEXT: v_max_f64 v[8:9], v[8:9], v[8:9]
-; GFX7-GISEL-NEXT: v_max_f64 v[10:11], v[10:11], v[10:11]
-; GFX7-GISEL-NEXT: v_max_f64 v[12:13], v[12:13], v[12:13]
-; GFX7-GISEL-NEXT: v_max_f64 v[14:15], v[14:15], v[14:15]
-; GFX7-GISEL-NEXT: v_max_f64 v[16:17], v[16:17], v[16:17]
-; GFX7-GISEL-NEXT: v_max_f64 v[18:19], v[18:19], v[18:19]
-; GFX7-GISEL-NEXT: v_max_f64 v[20:21], v[20:21], v[20:21]
-; GFX7-GISEL-NEXT: v_max_f64 v[22:23], v[22:23], v[22:23]
-; GFX7-GISEL-NEXT: v_max_f64 v[24:25], v[24:25], v[24:25]
-; GFX7-GISEL-NEXT: v_max_f64 v[26:27], v[26:27], v[26:27]
-; GFX7-GISEL-NEXT: v_max_f64 v[28:29], v[28:29], v[28:29]
; GFX7-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
-; GFX7-GISEL-NEXT: v_min_f64 v[2:3], v[4:5], v[6:7]
-; GFX7-GISEL-NEXT: v_min_f64 v[4:5], v[8:9], v[10:11]
-; GFX7-GISEL-NEXT: v_min_f64 v[6:7], v[12:13], v[14:15]
-; GFX7-GISEL-NEXT: v_min_f64 v[8:9], v[16:17], v[18:19]
-; GFX7-GISEL-NEXT: v_min_f64 v[10:11], v[20:21], v[22:23]
-; GFX7-GISEL-NEXT: v_min_f64 v[12:13], v[24:25], v[26:27]
+; GFX7-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[4:5]
+; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[6:7], v[6:7]
+; GFX7-GISEL-NEXT: v_max_f64 v[6:7], v[10:11], v[10:11]
+; GFX7-GISEL-NEXT: v_max_f64 v[10:11], v[14:15], v[14:15]
+; GFX7-GISEL-NEXT: v_max_f64 v[14:15], v[18:19], v[18:19]
+; GFX7-GISEL-NEXT: v_max_f64 v[18:19], v[22:23], v[22:23]
+; GFX7-GISEL-NEXT: v_max_f64 v[22:23], v[26:27], v[26:27]
+; GFX7-GISEL-NEXT: v_min_f64 v[2:3], v[2:3], v[4:5]
+; GFX7-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[8:9]
+; GFX7-GISEL-NEXT: v_max_f64 v[8:9], v[12:13], v[12:13]
+; GFX7-GISEL-NEXT: v_max_f64 v[12:13], v[16:17], v[16:17]
+; GFX7-GISEL-NEXT: v_max_f64 v[16:17], v[20:21], v[20:21]
+; GFX7-GISEL-NEXT: v_max_f64 v[20:21], v[24:25], v[24:25]
+; GFX7-GISEL-NEXT: v_max_f64 v[24:25], v[28:29], v[28:29]
; GFX7-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
+; GFX7-GISEL-NEXT: v_min_f64 v[4:5], v[4:5], v[6:7]
+; GFX7-GISEL-NEXT: v_min_f64 v[6:7], v[8:9], v[10:11]
+; GFX7-GISEL-NEXT: v_min_f64 v[8:9], v[12:13], v[14:15]
+; GFX7-GISEL-NEXT: v_min_f64 v[10:11], v[16:17], v[18:19]
+; GFX7-GISEL-NEXT: v_min_f64 v[12:13], v[20:21], v[22:23]
; GFX7-GISEL-NEXT: v_min_f64 v[2:3], v[4:5], v[6:7]
; GFX7-GISEL-NEXT: v_min_f64 v[4:5], v[8:9], v[10:11]
; GFX7-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
; GFX7-GISEL-NEXT: s_waitcnt vmcnt(0)
-; GFX7-GISEL-NEXT: v_max_f64 v[30:31], v[30:31], v[30:31]
-; GFX7-GISEL-NEXT: v_min_f64 v[14:15], v[28:29], v[30:31]
+; GFX7-GISEL-NEXT: v_max_f64 v[26:27], v[30:31], v[30:31]
+; GFX7-GISEL-NEXT: v_min_f64 v[14:15], v[24:25], v[26:27]
; GFX7-GISEL-NEXT: v_min_f64 v[6:7], v[12:13], v[14:15]
; GFX7-GISEL-NEXT: v_min_f64 v[2:3], v[4:5], v[6:7]
; GFX7-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
@@ -3589,33 +3589,33 @@ define double @test_vector_reduce_fmin_v16double(<16 x double> %v) {
; GFX8-GISEL-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX8-GISEL-NEXT: v_max_f64 v[0:1], v[0:1], v[0:1]
; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[2:3], v[2:3]
-; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[4:5], v[4:5]
-; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[6:7], v[6:7]
-; GFX8-GISEL-NEXT: v_max_f64 v[8:9], v[8:9], v[8:9]
-; GFX8-GISEL-NEXT: v_max_f64 v[10:11], v[10:11], v[10:11]
-; GFX8-GISEL-NEXT: v_max_f64 v[12:13], v[12:13], v[12:13]
-; GFX8-GISEL-NEXT: v_max_f64 v[14:15], v[14:15], v[14:15]
-; GFX8-GISEL-NEXT: v_max_f64 v[16:17], v[16:17], v[16:17]
-; GFX8-GISEL-NEXT: v_max_f64 v[18:19], v[18:19], v[18:19]
-; GFX8-GISEL-NEXT: v_max_f64 v[20:21], v[20:21], v[20:21]
-; GFX8-GISEL-NEXT: v_max_f64 v[22:23], v[22:23], v[22:23]
-; GFX8-GISEL-NEXT: v_max_f64 v[24:25], v[24:25], v[24:25]
-; GFX8-GISEL-NEXT: v_max_f64 v[26:27], v[26:27], v[26:27]
-; GFX8-GISEL-NEXT: v_max_f64 v[28:29], v[28:29], v[28:29]
; GFX8-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
-; GFX8-GISEL-NEXT: v_min_f64 v[2:3], v[4:5], v[6:7]
-; GFX8-GISEL-NEXT: v_min_f64 v[4:5], v[8:9], v[10:11]
-; GFX8-GISEL-NEXT: v_min_f64 v[6:7], v[12:13], v[14:15]
-; GFX8-GISEL-NEXT: v_min_f64 v[8:9], v[16:17], v[18:19]
-; GFX8-GISEL-NEXT: v_min_f64 v[10:11], v[20:21], v[22:23]
-; GFX8-GISEL-NEXT: v_min_f64 v[12:13], v[24:25], v[26:27]
+; GFX8-GISEL-NEXT: v_max_f64 v[2:3], v[4:5], v[4:5]
+; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[6:7], v[6:7]
+; GFX8-GISEL-NEXT: v_max_f64 v[6:7], v[10:11], v[10:11]
+; GFX8-GISEL-NEXT: v_max_f64 v[10:11], v[14:15], v[14:15]
+; GFX8-GISEL-NEXT: v_max_f64 v[14:15], v[18:19], v[18:19]
+; GFX8-GISEL-NEXT: v_max_f64 v[18:19], v[22:23], v[22:23]
+; GFX8-GISEL-NEXT: v_max_f64 v[22:23], v[26:27], v[26:27]
+; GFX8-GISEL-NEXT: v_min_f64 v[2:3], v[2:3], v[4:5]
+; GFX8-GISEL-NEXT: v_max_f64 v[4:5], v[8:9], v[8:9]
+; GFX8-GISEL-NEXT: v_max_f64 v[8:9], v[12:13], v[12:13]
+; GFX8-GISEL-NEXT: v_max_f64 v[12:13], v[16:17], v[16:17]
+; GFX8-GISEL-NEXT: v_max_f64 v[16:17], v[20:21], v[20:21]
+; GFX8-GISEL-NEXT: v_max_f64 v[20:21], v[24:25], v[24:25]
+; GFX8-GISEL-NEXT: v_max_f64 v[24:25], v[28:29], v[28:29]
; GFX8-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
+; GFX8-GISEL-NEXT: v_min_f64 v[4:5], v[4:5], v[6:7]
+; GFX8-GISEL-NEXT: v_min_f64 v[6:7], v[8:9], v[10:11]
+; GFX8-GISEL-NEXT: v_min_f64 v[8:9], v[12:13], v[14:15]
+; GFX8-GISEL-NEXT: v_min_f64 v[10:11], v[16:17], v[18:19]
+; GFX8-GISEL-NEXT: v_min_f64 v[12:13], v[20:21], v[22:23]
; GFX8-GISEL-NEXT: v_min_f64 v[2:3], v[4:5], v[6:7]
; GFX8-GISEL-NEXT: v_min_f64 v[4:5], v[8:9], v[10:11]
; GFX8-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
; GFX8-GISEL-NEXT: s_waitcnt vmcnt(0)
-; GFX8-GISEL-NEXT: v_max_f64 v[30:31], v[30:31], v[30:31]
-; GFX8-GISEL-NEXT: v_min_f64 v[14:15], v[28:29], v[30:31]
+; GFX8-GISEL-NEXT: v_max_f64 v[26:27], v[30:31], v[30:31]
+; GFX8-GISEL-NEXT: v_min_f64 v[14:15], v[24:25], v[26:27]
; GFX8-GISEL-NEXT: v_min_f64 v[6:7], v[12:13], v[14:15]
; GFX8-GISEL-NEXT: v_min_f64 v[2:3], v[4:5], v[6:7]
; GFX8-GISEL-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
diff --git a/llvm/test/CodeGen/AMDGPU/vector-reduce-fminimum.ll b/llvm/test/CodeGen/AMDGPU/vector-reduce-fminimum.ll
index 312fb59c0d9b5..aa8e4f749fe2e 100644
--- a/llvm/test/CodeGen/AMDGPU/vector-reduce-fminimum.ll
+++ b/llvm/test/CodeGen/AMDGPU/vector-reduce-fminimum.ll
@@ -2804,136 +2804,136 @@ define double @test_vector_reduce_fminimum_v16double(<16 x double> %v) {
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX7-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX7-NEXT: v_min_f64 v[32:33], v[8:9], v[24:25]
; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[8:9], v[24:25]
-; GFX7-NEXT: v_min_f64 v[8:9], v[0:1], v[16:17]
+; GFX7-NEXT: v_min_f64 v[8:9], v[8:9], v[24:25]
; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[16:17]
-; GFX7-NEXT: v_min_f64 v[0:1], v[12:13], v[28:29]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[12:13], v[28:29]
-; GFX7-NEXT: v_min_f64 v[12:13], v[10:11], v[26:27]
+; GFX7-NEXT: v_min_f64 v[24:25], v[0:1], v[16:17]
+; GFX7-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[10:11], v[26:27]
-; GFX7-NEXT: v_min_f64 v[10:11], v[2:3], v[18:19]
; GFX7-NEXT: v_cmp_u_f64_e64 s[10:11], v[2:3], v[18:19]
-; GFX7-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX7-NEXT: v_min_f64 v[2:3], v[4:5], v[20:21]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[12:13], v[4:5], v[20:21]
-; GFX7-NEXT: v_min_f64 v[4:5], v[6:7], v[22:23]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[14:15], v[6:7], v[22:23]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v13, v34, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v12, 0, s[8:9]
-; GFX7-NEXT: v_cndmask_b32_e64 v11, v11, v34, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[10:11]
-; GFX7-NEXT: v_cndmask_b32_e32 v13, v33, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v12, v32, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v9, v9, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[4:5]
-; GFX7-NEXT: v_min_f64 v[16:17], v[10:11], v[6:7]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[10:11], v[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v34, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[12:13]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v5, v34, s[14:15]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[14:15]
-; GFX7-NEXT: v_min_f64 v[10:11], v[2:3], v[0:1]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[2:3], v[0:1]
-; GFX7-NEXT: v_min_f64 v[0:1], v[8:9], v[12:13]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[8:9], v[12:13]
-; GFX7-NEXT: v_cndmask_b32_e32 v3, v17, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v16, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[8:9]
+; GFX7-NEXT: v_min_f64 v[2:3], v[2:3], v[18:19]
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v9, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
+; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[12:13], v[28:29]
+; GFX7-NEXT: v_min_f64 v[8:9], v[12:13], v[28:29]
+; GFX7-NEXT: v_min_f64 v[12:13], v[4:5], v[20:21]
+; GFX7-NEXT: v_min_f64 v[20:21], v[10:11], v[26:27]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v25, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[10:11]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v24, 0, s[4:5]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[6:7], v[22:23]
+; GFX7-NEXT: v_cndmask_b32_e32 v9, v9, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v11, v13, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v10, v12, 0, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v13, v21, v16, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v12, v20, 0, s[8:9]
+; GFX7-NEXT: v_cndmask_b32_e64 v8, v8, 0, vcc
+; GFX7-NEXT: v_min_f64 v[6:7], v[6:7], v[22:23]
+; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[12:13]
+; GFX7-NEXT: v_min_f64 v[2:3], v[2:3], v[12:13]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[8:9], v[4:5], v[0:1]
+; GFX7-NEXT: v_min_f64 v[0:1], v[4:5], v[0:1]
+; GFX7-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[10:11], v[8:9]
+; GFX7-NEXT: v_min_f64 v[8:9], v[10:11], v[8:9]
+; GFX7-NEXT: v_cndmask_b32_e32 v3, v3, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v1, v1, v16, s[8:9]
; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[8:9]
; GFX7-NEXT: s_waitcnt vmcnt(0)
-; GFX7-NEXT: v_min_f64 v[6:7], v[14:15], v[30:31]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[14:15], v[30:31]
-; GFX7-NEXT: v_cndmask_b32_e64 v7, v7, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
-; GFX7-NEXT: v_min_f64 v[8:9], v[4:5], v[6:7]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[4:5], v[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v5, v11, v34, s[6:7]
-; GFX7-NEXT: v_cndmask_b32_e64 v4, v10, 0, s[6:7]
-; GFX7-NEXT: v_min_f64 v[6:7], v[0:1], v[4:5]
-; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[4:5]
-; GFX7-NEXT: v_cndmask_b32_e32 v1, v9, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
-; GFX7-NEXT: v_min_f64 v[4:5], v[2:3], v[0:1]
-; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[0:1]
-; GFX7-NEXT: v_cndmask_b32_e64 v1, v7, v34, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v6, 0, s[4:5]
-; GFX7-NEXT: v_cndmask_b32_e32 v3, v5, v34, vcc
-; GFX7-NEXT: v_cndmask_b32_e64 v2, v4, 0, vcc
-; GFX7-NEXT: v_min_f64 v[4:5], v[0:1], v[2:3]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[14:15], v[30:31]
+; GFX7-NEXT: v_min_f64 v[12:13], v[14:15], v[30:31]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v13, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v12, 0, s[6:7]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[6:7], v[6:7], v[4:5]
+; GFX7-NEXT: v_min_f64 v[4:5], v[6:7], v[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v7, v9, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v6, v8, 0, s[4:5]
+; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[6:7]
+; GFX7-NEXT: v_min_f64 v[0:1], v[0:1], v[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX7-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX7-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[4:5]
+; GFX7-NEXT: v_min_f64 v[2:3], v[2:3], v[4:5]
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX7-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
; GFX7-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[2:3]
-; GFX7-NEXT: v_cndmask_b32_e64 v0, v4, 0, vcc
-; GFX7-NEXT: v_cndmask_b32_e32 v1, v5, v34, vcc
+; GFX7-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
+; GFX7-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX7-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
; GFX7-NEXT: s_setpc_b64 s[30:31]
;
; GFX8-LABEL: test_vector_reduce_fminimum_v16double:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX8-NEXT: buffer_load_dword v31, off, s[0:3], s32
-; GFX8-NEXT: v_min_f64 v[32:33], v[8:9], v[24:25]
; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[8:9], v[24:25]
-; GFX8-NEXT: v_min_f64 v[8:9], v[0:1], v[16:17]
+; GFX8-NEXT: v_min_f64 v[8:9], v[8:9], v[24:25]
; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[16:17]
-; GFX8-NEXT: v_min_f64 v[0:1], v[12:13], v[28:29]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[12:13], v[28:29]
-; GFX8-NEXT: v_min_f64 v[12:13], v[10:11], v[26:27]
+; GFX8-NEXT: v_min_f64 v[24:25], v[0:1], v[16:17]
+; GFX8-NEXT: v_mov_b32_e32 v16, 0x7ff80000
+; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[4:5], v[20:21]
; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[10:11], v[26:27]
-; GFX8-NEXT: v_min_f64 v[10:11], v[2:3], v[18:19]
; GFX8-NEXT: v_cmp_u_f64_e64 s[10:11], v[2:3], v[18:19]
-; GFX8-NEXT: v_mov_b32_e32 v34, 0x7ff80000
-; GFX8-NEXT: v_min_f64 v[2:3], v[4:5], v[20:21]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[12:13], v[4:5], v[20:21]
-; GFX8-NEXT: v_min_f64 v[4:5], v[6:7], v[22:23]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[14:15], v[6:7], v[22:23]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v13, v34, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v12, 0, s[8:9]
-; GFX8-NEXT: v_cndmask_b32_e64 v11, v11, v34, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e64 v10, v10, 0, s[10:11]
-; GFX8-NEXT: v_cndmask_b32_e32 v13, v33, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v12, v32, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v9, v9, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, 0, s[4:5]
-; GFX8-NEXT: v_min_f64 v[16:17], v[10:11], v[6:7]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[10:11], v[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v34, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[12:13]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v5, v34, s[14:15]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[14:15]
-; GFX8-NEXT: v_min_f64 v[10:11], v[2:3], v[0:1]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[2:3], v[0:1]
-; GFX8-NEXT: v_min_f64 v[0:1], v[8:9], v[12:13]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[8:9], v[12:13]
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v17, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v16, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v34, s[8:9]
+; GFX8-NEXT: v_min_f64 v[2:3], v[2:3], v[18:19]
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v9, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
+; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[12:13], v[28:29]
+; GFX8-NEXT: v_min_f64 v[8:9], v[12:13], v[28:29]
+; GFX8-NEXT: v_min_f64 v[12:13], v[4:5], v[20:21]
+; GFX8-NEXT: v_min_f64 v[20:21], v[10:11], v[26:27]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v25, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[10:11]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v24, 0, s[4:5]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[6:7], v[22:23]
+; GFX8-NEXT: v_cndmask_b32_e32 v9, v9, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v11, v13, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v10, v12, 0, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v13, v21, v16, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v12, v20, 0, s[8:9]
+; GFX8-NEXT: v_cndmask_b32_e64 v8, v8, 0, vcc
+; GFX8-NEXT: v_min_f64 v[6:7], v[6:7], v[22:23]
+; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[12:13]
+; GFX8-NEXT: v_min_f64 v[2:3], v[2:3], v[12:13]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[8:9], v[4:5], v[0:1]
+; GFX8-NEXT: v_min_f64 v[0:1], v[4:5], v[0:1]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[10:11], v[8:9]
+; GFX8-NEXT: v_min_f64 v[8:9], v[10:11], v[8:9]
+; GFX8-NEXT: v_cndmask_b32_e32 v3, v3, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v1, v1, v16, s[8:9]
; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[8:9]
; GFX8-NEXT: s_waitcnt vmcnt(0)
-; GFX8-NEXT: v_min_f64 v[6:7], v[14:15], v[30:31]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[14:15], v[30:31]
-; GFX8-NEXT: v_cndmask_b32_e64 v7, v7, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v6, v6, 0, s[4:5]
-; GFX8-NEXT: v_min_f64 v[8:9], v[4:5], v[6:7]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[4:5], v[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v5, v11, v34, s[6:7]
-; GFX8-NEXT: v_cndmask_b32_e64 v4, v10, 0, s[6:7]
-; GFX8-NEXT: v_min_f64 v[6:7], v[0:1], v[4:5]
-; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[0:1], v[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v9, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v8, 0, vcc
-; GFX8-NEXT: v_min_f64 v[4:5], v[2:3], v[0:1]
-; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[2:3], v[0:1]
-; GFX8-NEXT: v_cndmask_b32_e64 v1, v7, v34, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v6, 0, s[4:5]
-; GFX8-NEXT: v_cndmask_b32_e32 v3, v5, v34, vcc
-; GFX8-NEXT: v_cndmask_b32_e64 v2, v4, 0, vcc
-; GFX8-NEXT: v_min_f64 v[4:5], v[0:1], v[2:3]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[14:15], v[30:31]
+; GFX8-NEXT: v_min_f64 v[12:13], v[14:15], v[30:31]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v13, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v12, 0, s[6:7]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[6:7], v[6:7], v[4:5]
+; GFX8-NEXT: v_min_f64 v[4:5], v[6:7], v[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v7, v9, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v6, v8, 0, s[4:5]
+; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[6:7]
+; GFX8-NEXT: v_min_f64 v[0:1], v[0:1], v[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v5, v5, v16, s[6:7]
+; GFX8-NEXT: v_cndmask_b32_e64 v4, v4, 0, s[6:7]
+; GFX8-NEXT: v_cmp_u_f64_e64 s[4:5], v[2:3], v[4:5]
+; GFX8-NEXT: v_min_f64 v[2:3], v[2:3], v[4:5]
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e64 v3, v3, v16, s[4:5]
+; GFX8-NEXT: v_cndmask_b32_e64 v2, v2, 0, s[4:5]
; GFX8-NEXT: v_cmp_u_f64_e32 vcc, v[0:1], v[2:3]
-; GFX8-NEXT: v_cndmask_b32_e64 v0, v4, 0, vcc
-; GFX8-NEXT: v_cndmask_b32_e32 v1, v5, v34, vcc
+; GFX8-NEXT: v_min_f64 v[0:1], v[0:1], v[2:3]
+; GFX8-NEXT: v_cndmask_b32_e64 v0, v0, 0, vcc
+; GFX8-NEXT: v_cndmask_b32_e32 v1, v1, v16, vcc
; GFX8-NEXT: s_setpc_b64 s[30:31]
;
; GFX9-LABEL: test_vector_reduce_fminimum_v16double:
diff --git a/llvm/test/CodeGen/AMDGPU/vector-reduce-mul.ll b/llvm/test/CodeGen/AMDGPU/vector-reduce-mul.ll
index ea563e99ba6e2..d3526720b4a21 100644
--- a/llvm/test/CodeGen/AMDGPU/vector-reduce-mul.ll
+++ b/llvm/test/CodeGen/AMDGPU/vector-reduce-mul.ll
@@ -3677,82 +3677,82 @@ define i64 @test_vector_reduce_mul_v16i64(<16 x i64> %v) {
; GFX7-SDAG-LABEL: test_vector_reduce_mul_v16i64:
; GFX7-SDAG: ; %bb.0: ; %entry
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX7-SDAG-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX7-SDAG-NEXT: v_mul_lo_u32 v19, v2, v19
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v32, v3, v18
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v31, v3, v18
; GFX7-SDAG-NEXT: v_mad_u64_u32 v[2:3], s[4:5], v2, v18, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v27, v10, v27
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v33, v11, v26
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[10:11], s[4:5], v10, v26, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v3, vcc, v3, v19
-; GFX7-SDAG-NEXT: v_add_i32_e32 v32, vcc, v3, v32
; GFX7-SDAG-NEXT: v_mul_lo_u32 v21, v4, v21
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v4, v20, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v23, v6, v23
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v26, v7, v22
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v13, v13, v28
+; GFX7-SDAG-NEXT: v_add_i32_e32 v3, vcc, v3, v19
+; GFX7-SDAG-NEXT: v_add_i32_e32 v18, vcc, v3, v31
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v3, v10, v27
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v19, v11, v26
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[10:11], s[4:5], v10, v26, 0
+; GFX7-SDAG-NEXT: v_add_i32_e32 v3, vcc, v11, v3
+; GFX7-SDAG-NEXT: v_add_i32_e32 v11, vcc, v3, v19
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v3, v0, v17
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v17, v1, v16
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v0, v16, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v11, v2, v11
+; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v1, v3
+; GFX7-SDAG-NEXT: v_add_i32_e32 v19, vcc, v1, v17
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v1, v8, v25
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v3, v9, v24
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v8, v24, 0
+; GFX7-SDAG-NEXT: buffer_load_dword v24, off, s[0:3], s32
+; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v9, v1
+; GFX7-SDAG-NEXT: v_add_i32_e32 v17, vcc, v1, v3
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v1, v6, v23
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v3, v7, v22
; GFX7-SDAG-NEXT: v_mad_u64_u32 v[6:7], s[4:5], v6, v22, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v5, v5, v20
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v20, v12, v29
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v29, v13, v28
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[12:13], s[4:5], v12, v28, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v11, vcc, v11, v27
-; GFX7-SDAG-NEXT: v_add_i32_e32 v11, vcc, v11, v33
-; GFX7-SDAG-NEXT: v_add_i32_e32 v4, vcc, v4, v21
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[18:19], s[4:5], v14, v30, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v7, vcc, v7, v23
-; GFX7-SDAG-NEXT: v_add_i32_e32 v23, vcc, v4, v5
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v2, v10, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v2, v2, v11
-; GFX7-SDAG-NEXT: v_add_i32_e32 v11, vcc, v13, v20
; GFX7-SDAG-NEXT: v_mul_lo_u32 v22, v15, v30
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v17, v0, v17
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v28, v1, v16
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v0, v16, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v25, v8, v25
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[15:16], s[4:5], v8, v24, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v21, vcc, v7, v26
-; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v1, v17
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[7:8], s[4:5], v6, v18, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v10, v32, v10
-; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v5, v2
-; GFX7-SDAG-NEXT: v_add_i32_e32 v11, vcc, v11, v29
-; GFX7-SDAG-NEXT: v_add_i32_e32 v5, vcc, v2, v10
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[15:16], s[4:5], v14, v30, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v23, v5, v20
+; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v7, v1
+; GFX7-SDAG-NEXT: v_add_i32_e32 v7, vcc, v1, v3
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v4, v20, 0
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[1:2], s[4:5], v2, v10, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v20, v18, v10
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[9:10], s[4:5], v0, v8, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v0, v0, v17
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[17:18], s[4:5], v6, v15, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v7, v7, v15
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v15, v12, v29
+; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v2, v11
+; GFX7-SDAG-NEXT: v_add_i32_e32 v11, vcc, v2, v20
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v8, v19, v8
+; GFX7-SDAG-NEXT: v_add_i32_e32 v0, vcc, v10, v0
+; GFX7-SDAG-NEXT: v_add_i32_e32 v0, vcc, v0, v8
; GFX7-SDAG-NEXT: s_waitcnt vmcnt(0)
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v13, v14, v31
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v14, v21, v18
-; GFX7-SDAG-NEXT: v_add_i32_e32 v13, vcc, v19, v13
-; GFX7-SDAG-NEXT: v_add_i32_e32 v13, vcc, v13, v22
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v6, v6, v13
-; GFX7-SDAG-NEXT: v_add_i32_e32 v13, vcc, v1, v28
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v1, v9, v24
-; GFX7-SDAG-NEXT: v_add_i32_e32 v9, vcc, v16, v25
-; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v8, v6
-; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v9, v1
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v6, v0, v1
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v0, v15, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v8, vcc, v2, v14
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v9, v3, v11
-; GFX7-SDAG-NEXT: v_add_i32_e32 v6, vcc, v1, v6
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[1:2], s[4:5], v3, v12, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v3, v23, v12
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v10, v13, v15
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v8, v4, v8
-; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v2, v9
-; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v2, v3
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v9, v0, v2
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[2:3], s[4:5], v0, v1, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v0, vcc, v6, v10
-; GFX7-SDAG-NEXT: v_add_i32_e32 v6, vcc, v3, v9
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v4, v7, 0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v9, v0, v1
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v0, v5, v7
-; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v4, v8
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v5, v14, v24
+; GFX7-SDAG-NEXT: v_add_i32_e32 v14, vcc, v4, v21
+; GFX7-SDAG-NEXT: v_add_i32_e32 v5, vcc, v16, v5
+; GFX7-SDAG-NEXT: v_add_i32_e32 v5, vcc, v5, v22
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v6, v6, v5
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v12, v28, 0
+; GFX7-SDAG-NEXT: v_add_i32_e32 v12, vcc, v14, v23
+; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v18, v6
+; GFX7-SDAG-NEXT: v_add_i32_e32 v5, vcc, v5, v15
+; GFX7-SDAG-NEXT: v_add_i32_e32 v5, vcc, v5, v13
+; GFX7-SDAG-NEXT: v_add_i32_e32 v6, vcc, v2, v7
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v5, v3, v5
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[2:3], s[4:5], v3, v4, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v4, v12, v4
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v6, v1, v6
+; GFX7-SDAG-NEXT: v_add_i32_e32 v3, vcc, v3, v5
+; GFX7-SDAG-NEXT: v_add_i32_e32 v3, vcc, v3, v4
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v5, v9, v3
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v9, v2, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v2, v0, v2
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v0, v11, v17
+; GFX7-SDAG-NEXT: v_add_i32_e32 v7, vcc, v4, v5
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v1, v17, 0
+; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v7, v2
+; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v5, v6
; GFX7-SDAG-NEXT: v_add_i32_e32 v0, vcc, v1, v0
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v4, v2, v0
-; GFX7-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v2, v3, 0
-; GFX7-SDAG-NEXT: v_add_i32_e32 v2, vcc, v6, v9
-; GFX7-SDAG-NEXT: v_mul_lo_u32 v2, v2, v3
-; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v1, v4
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v5, v3, v0
+; GFX7-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v3, v4, 0
+; GFX7-SDAG-NEXT: v_mul_lo_u32 v2, v2, v4
+; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v1, v5
; GFX7-SDAG-NEXT: v_add_i32_e32 v1, vcc, v1, v2
; GFX7-SDAG-NEXT: s_setpc_b64 s[30:31]
;
@@ -3811,82 +3811,82 @@ define i64 @test_vector_reduce_mul_v16i64(<16 x i64> %v) {
; GFX8-SDAG-LABEL: test_vector_reduce_mul_v16i64:
; GFX8-SDAG: ; %bb.0: ; %entry
; GFX8-SDAG-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-SDAG-NEXT: buffer_load_dword v31, off, s[0:3], s32
; GFX8-SDAG-NEXT: v_mul_lo_u32 v19, v2, v19
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v32, v3, v18
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v31, v3, v18
; GFX8-SDAG-NEXT: v_mad_u64_u32 v[2:3], s[4:5], v2, v18, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v27, v10, v27
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v33, v11, v26
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[10:11], s[4:5], v10, v26, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v3, vcc, v3, v19
-; GFX8-SDAG-NEXT: v_add_u32_e32 v32, vcc, v3, v32
; GFX8-SDAG-NEXT: v_mul_lo_u32 v21, v4, v21
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v4, v20, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v23, v6, v23
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v26, v7, v22
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v13, v13, v28
+; GFX8-SDAG-NEXT: v_add_u32_e32 v3, vcc, v3, v19
+; GFX8-SDAG-NEXT: v_add_u32_e32 v18, vcc, v3, v31
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v3, v10, v27
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v19, v11, v26
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[10:11], s[4:5], v10, v26, 0
+; GFX8-SDAG-NEXT: v_add_u32_e32 v3, vcc, v11, v3
+; GFX8-SDAG-NEXT: v_add_u32_e32 v11, vcc, v3, v19
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v3, v0, v17
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v17, v1, v16
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v0, v16, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v11, v2, v11
+; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v1, v3
+; GFX8-SDAG-NEXT: v_add_u32_e32 v19, vcc, v1, v17
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v1, v8, v25
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v3, v9, v24
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[8:9], s[4:5], v8, v24, 0
+; GFX8-SDAG-NEXT: buffer_load_dword v24, off, s[0:3], s32
+; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v9, v1
+; GFX8-SDAG-NEXT: v_add_u32_e32 v17, vcc, v1, v3
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v1, v6, v23
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v3, v7, v22
; GFX8-SDAG-NEXT: v_mad_u64_u32 v[6:7], s[4:5], v6, v22, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v5, v5, v20
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v20, v12, v29
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v29, v13, v28
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[12:13], s[4:5], v12, v28, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v11, vcc, v11, v27
-; GFX8-SDAG-NEXT: v_add_u32_e32 v11, vcc, v11, v33
-; GFX8-SDAG-NEXT: v_add_u32_e32 v4, vcc, v4, v21
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[18:19], s[4:5], v14, v30, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v7, vcc, v7, v23
-; GFX8-SDAG-NEXT: v_add_u32_e32 v23, vcc, v4, v5
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v2, v10, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v2, v2, v11
-; GFX8-SDAG-NEXT: v_add_u32_e32 v11, vcc, v13, v20
; GFX8-SDAG-NEXT: v_mul_lo_u32 v22, v15, v30
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v17, v0, v17
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v28, v1, v16
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v0, v16, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v25, v8, v25
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[15:16], s[4:5], v8, v24, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v21, vcc, v7, v26
-; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v1, v17
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[7:8], s[4:5], v6, v18, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v10, v32, v10
-; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v5, v2
-; GFX8-SDAG-NEXT: v_add_u32_e32 v11, vcc, v11, v29
-; GFX8-SDAG-NEXT: v_add_u32_e32 v5, vcc, v2, v10
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[15:16], s[4:5], v14, v30, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v23, v5, v20
+; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v7, v1
+; GFX8-SDAG-NEXT: v_add_u32_e32 v7, vcc, v1, v3
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v4, v20, 0
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[1:2], s[4:5], v2, v10, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v20, v18, v10
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[9:10], s[4:5], v0, v8, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v0, v0, v17
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[17:18], s[4:5], v6, v15, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v7, v7, v15
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v15, v12, v29
+; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v2, v11
+; GFX8-SDAG-NEXT: v_add_u32_e32 v11, vcc, v2, v20
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v8, v19, v8
+; GFX8-SDAG-NEXT: v_add_u32_e32 v0, vcc, v10, v0
+; GFX8-SDAG-NEXT: v_add_u32_e32 v0, vcc, v0, v8
; GFX8-SDAG-NEXT: s_waitcnt vmcnt(0)
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v13, v14, v31
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v14, v21, v18
-; GFX8-SDAG-NEXT: v_add_u32_e32 v13, vcc, v19, v13
-; GFX8-SDAG-NEXT: v_add_u32_e32 v13, vcc, v13, v22
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v6, v6, v13
-; GFX8-SDAG-NEXT: v_add_u32_e32 v13, vcc, v1, v28
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v1, v9, v24
-; GFX8-SDAG-NEXT: v_add_u32_e32 v9, vcc, v16, v25
-; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v8, v6
-; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v9, v1
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v6, v0, v1
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v0, v15, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v8, vcc, v2, v14
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v9, v3, v11
-; GFX8-SDAG-NEXT: v_add_u32_e32 v6, vcc, v1, v6
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[1:2], s[4:5], v3, v12, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v3, v23, v12
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v10, v13, v15
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v8, v4, v8
-; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v2, v9
-; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v2, v3
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v9, v0, v2
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[2:3], s[4:5], v0, v1, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v0, vcc, v6, v10
-; GFX8-SDAG-NEXT: v_add_u32_e32 v6, vcc, v3, v9
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v4, v7, 0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v9, v0, v1
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v0, v5, v7
-; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v4, v8
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v5, v14, v24
+; GFX8-SDAG-NEXT: v_add_u32_e32 v14, vcc, v4, v21
+; GFX8-SDAG-NEXT: v_add_u32_e32 v5, vcc, v16, v5
+; GFX8-SDAG-NEXT: v_add_u32_e32 v5, vcc, v5, v22
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v6, v6, v5
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v12, v28, 0
+; GFX8-SDAG-NEXT: v_add_u32_e32 v12, vcc, v14, v23
+; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v18, v6
+; GFX8-SDAG-NEXT: v_add_u32_e32 v5, vcc, v5, v15
+; GFX8-SDAG-NEXT: v_add_u32_e32 v5, vcc, v5, v13
+; GFX8-SDAG-NEXT: v_add_u32_e32 v6, vcc, v2, v7
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v5, v3, v5
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[2:3], s[4:5], v3, v4, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v4, v12, v4
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v6, v1, v6
+; GFX8-SDAG-NEXT: v_add_u32_e32 v3, vcc, v3, v5
+; GFX8-SDAG-NEXT: v_add_u32_e32 v3, vcc, v3, v4
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v5, v9, v3
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[3:4], s[4:5], v9, v2, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v2, v0, v2
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v0, v11, v17
+; GFX8-SDAG-NEXT: v_add_u32_e32 v7, vcc, v4, v5
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[4:5], s[4:5], v1, v17, 0
+; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v7, v2
+; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v5, v6
; GFX8-SDAG-NEXT: v_add_u32_e32 v0, vcc, v1, v0
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v4, v2, v0
-; GFX8-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v2, v3, 0
-; GFX8-SDAG-NEXT: v_add_u32_e32 v2, vcc, v6, v9
-; GFX8-SDAG-NEXT: v_mul_lo_u32 v2, v2, v3
-; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v1, v4
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v5, v3, v0
+; GFX8-SDAG-NEXT: v_mad_u64_u32 v[0:1], s[4:5], v3, v4, 0
+; GFX8-SDAG-NEXT: v_mul_lo_u32 v2, v2, v4
+; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v1, v5
; GFX8-SDAG-NEXT: v_add_u32_e32 v1, vcc, v1, v2
; GFX8-SDAG-NEXT: s_setpc_b64 s[30:31]
;
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll
index 735b8e0f74089..8b3aa57b1faf5 100644
--- a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll
@@ -119,6 +119,10 @@ define amdgpu_cs void @max_11_vgprs_branch(ptr addrspace(1) %p, i32 %tmp) "amdgp
; CHECK-NEXT: global_store_b32 v[0:1], v10, off scope:SCOPE_SYS
; CHECK-NEXT: s_wait_storecnt 0x0
; CHECK-NEXT: scratch_load_b32 v0, off, off offset:16 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: ; implicit-def: $vgpr2
+; CHECK-NEXT: ; kill: killed $vgpr2
+; CHECK-NEXT: ; implicit-def: $vgpr2
+; CHECK-NEXT: ; kill: killed $vgpr2
; CHECK-NEXT: s_wait_loadcnt 0x0
; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
; CHECK-NEXT: s_wait_storecnt 0x0
@@ -146,10 +150,6 @@ define amdgpu_cs void @max_11_vgprs_branch(ptr addrspace(1) %p, i32 %tmp) "amdgp
; CHECK-NEXT: s_wait_loadcnt 0x0
; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: ; implicit-def: $vgpr0
-; CHECK-NEXT: ; kill: killed $vgpr0
-; CHECK-NEXT: ; implicit-def: $vgpr0
-; CHECK-NEXT: ; kill: killed $vgpr0
; CHECK-NEXT: ; implicit-def: $vgpr0_vgpr1
; CHECK-NEXT: .LBB1_2: ; %Flow
; CHECK-NEXT: s_and_not1_saveexec_b32 s0, s0
More information about the llvm-branch-commits
mailing list